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ABSTRACT 

This is the first volume of a. report on a study that 

(1) investigated the "normative structure" (the governance system) of 
knowledge production and utilization (KPU) activities in education r 

(2) developed an analytical framework through which to understand how 
formal policy acts as a "regulator" of activities in KPU, (3) 
described major policies of significance to KPU and how they 
influence the governance of KPU processes in 10 different, 
representative case studies, and (tt) made recommendations that would 
help in the design of a monitoring program. This volume of the report 
describes the conduct of the study, the methodological -framework that 
was developed, and the recommendations' that were inferred. It also 
contains an annotated bibliography of various topics^ i^^^^ by the 
study. Appended are a bibliographic essay of nera|t:systems theory 
and KPU in educatijfjgt^j^ aad a discussion of K|g^i|i^s an^^s^ca : 
system. (RC) ^ ' 
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PREFACE 



This is one of two volumes constituting the final report of an 18-month 
study entitled "The Normative Structure of Knowledge Production andJiJtili- 
zation in Education/' performed by Stanford Research Institute under con- 
tract to the National Institute of Education (NIE), Reports and working 
papers produced as part of this study include: 

— 9- -0 i -W Ma rkley-,---^Th e--No rma ti-ve~S tr-u^a^ .. 

duction and Utilization; Interim Report" (December 1974). 

• T. Mandel^ "Development and Application of the Analytical 
Framework" (December 1974) . 

• R. Prewitt^ "Functional Subsystems for Curriculum Reform" 
(December 1974). 

• A. Zink^ "Use of Mind or Behavior Modifying Techniques in 
Education" (December 1974). 

The central objective of the study was to develop an analytical frame- 
work for describing the governance system that influences knowledge produc- 
tion and utilization (KPU) activities in education. In this volume we 
describe the analytical framework^ explain how it was developed,* and dis- 
cuss its implications for a research and development (R&D) monitoring pro- 
gram. 

The second volume demonstrates the use of the analytic framework and 
describes the substantive findings that resulted when the framework was 
applied to ten different case study topics. The titles of these ten topics 
are: 

I, National Institute of Education (NIE) Allocation Policy 

II. Federal Procurement Policy and Knowledge Production and 
Utilization in Education 
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III. Assessing the Impact of Policies that Control the Avail- 
ability of Information 

IV. The Far West Laboratory as a Research and Development 
Performer 

V. Minicourses as an Example of Policies Affecting the 
Dissemination/Utilization of a Successful R&D Product 

VI. The School Mathematics Study Group (SMSG) Project as an 
Example of Policies Affecting the Dissemination/Utiliza- 
tion of an R&D Product 

VII. ESEA Title III Teacher Initiated Innovation Program: An 
Example of Policies Interfacing Levels of Government 

VIII. The Governance of Knowledge Production and Utilization 
in Intermediate Service Agencies: Boards of Cooperative 



Educational Services in Colorado and New York 

IX. Policies Affecting the Results of the Federally Sponsored 
Pilot State Dissemination Program in South Carolina: 
1970-73 

X, An Approach to Monitoring the Role of Government Policies 
in the Process for Selection and Evaluation of New In- 
structional Materials 

This study is one of a series sponsored by NIE's R&D Systems Support 
Division in response to the recommendations of exploratory position papers 
such as "Building Capacity for Renewal and Reform" (Task Force on Resources 
Planning and Analysis, 1973) and "Modelling a National Educational R&D 
System" (Churchill, 1974). Under the direction of Dr. Ward Mason, the 
R&D Systems Support Division has responded to NIE's legislative mandate 
to help build an effective R&D system in education by pursuing three in- 
terrelated goals: 

1. To develop a monitoring "system that will lead to a sys- 
tematic data base concerning educational knowledge pro- 
duction and utilization. 



2, To initiate a series of studies that will: 

a. Develop models of the educational l^U process that 
lead to a greater understanding of applied system 
dynamics; 
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b. Assess the status of the R&D system^ the educational 
system, and the changes occurring in those systems; 

c. Identify problems and areas of weakness or imbalance 
in the educational KPU system for which NIE support 
activities are needed; 

d. Be useful to NIE policymakers^ to the R&D and edu- 
cational communities, and to the general public. 

3. To design and manage specific programs for strengthening 
the educational KPU system. 

Related investigations supported by the R&D Systems Support Division 
include: 

• William Paisl ey and assoclaLes at S L auIord Un iv e r s it y^ — pre* 

paring the first two editions of a Databook and a separate 
technical report in which they will analyze existing data 

bases and make recommendations for the development of a more 
coherent system of statistical indicators regarding the 
status of KPU in education. 

• Rolf Lemming at NIE, conducting a survey of various insti- 
tutional performers of KPU in education. 

• Michael Radnor at Northwestern University, studying R&D sys- 
tems in such areas as agriculture, aerospace, and defense 

to deduce applicable principles for R&D management in educa- 
tion. 

• David Clark and Egon Cuba at Indiana University, studying 
the KPU-related roles of departments, schools, and colleges 
of education. 

These studies constitute a set of preliminary ''predesign'* studies 
that should illuminate the actual design of a monitoring system at a later 
date. 
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GLOSSARY 



Agent 



Ana ly t i c framework 
Conceptual subsystems 
Configuration 



Field of analysis 
Focus of analysis 
Functional subsystems 



Infrastructure 



KPU 



A legally responsible entity, one or more persons 
concerned with any phase of the KPU system, in- 
cluding individuals , teams of persons , and 
institutions that act in a relatively unitary 
fashion. The terms agency and actor may also be 
used where convenient to distinguish the institu- 
tion from the institution's representative person. 

A conceptual structure that guides inquiry into 
and analysis of KPU in education. 

Subsystems that follow from a particular way of 
conceptualizing the system under study. 

An image or description of a portion (or a whole) 
of the KPU infrastructure as developed from a 
particular perspective. A configuration is an 
arrangement of a set of features and distinctions 
made by an analyst who guides his inquiry with a 
particular purpose. 

The environment and ecology of the focus of analy- 
sis. 

The agents, activities, policies, and resources of 
central concern to a specific purpose of analysis. 

Subsystems that emerge empirically to accomplish a 
particular function in the larger system under 
study. 

The interrelated elements through which the process 
of knowledge production and utilization in education 
takes place. 

The sum of innovative activities ranging from basic 
research to installation of new practices through 
which new knowledge is produced and used in educa- 
tion. The term is broader than simply research and 
development, incorporating as well the linkage of 
the research and development activities and the 
utilization of their products. Such activities as 
evaluation, demonstration, disS'Smination, diffusion, 

xvii . ■ 
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policy studies, or policy research are to be con- 
sidered as components of the four basic KPU ac- 
tivity categories of research, development, linkage, 
and utilization. 

Normative structure The set of values and principles (both formal poli- 

~ "cles^Tid-rtnf ormal norms-)- that -guide- behavior con-- 

cerned with the production, dissemination, and 
utilization of educational knowledge. 

Purposive activities A set of actions that reflect or build toward the 

intent of/ some formal statement of goals or norms, 
or what we call formal policy. (See goal oriented 
activity . ) 

Resources The matter, energy, and information needed to engage 

in a particular activity. Resources consumed and/or 
transformed by agents into other resources include 
money, educational products, information, individual 
skills, and the like. 



In our discussion of Phase 2 of the project we abandoned a number of 
terms used in our discussion of Phase 1. We did this for the following 
reasons: either we could replace them with new terms that more closely 
describe the concept; or we did not mention the concept itself in our 
discussion of Phase 2. 

To assist the reader in understanding the discussion of Phase 1 we 
list and explain these abandoned terms below. The terms have been segre-. 
gated from the rest of the glossary to emphasize the fact that they have 
been abandoned or replaced by new terminology in the discussion of Phase 
2 of the project. 

Flow The movement of resources and influences through 

the EKPU system. 

Goal oriented Activity evaluated against and modified to serve 

Activity some intent. (See purposive activity>)' 

Process loop A prescribed and time-sequenced series of actions 

incorporating planning and evaluation and directed 
toward the accomplishment of an objective. 



Public Policy 
Regulators (PPRs) 



Regulators 



Regulatory agent 



Directives that are codified and have a legal basis. 
Examples include statutory law, codes of ethics, 
certification evaluation and planning requirements, 
formal incentives, and budgetary priorities. Ex- 
cluded from this subset are informal norms, unwrit- 
ten procedural conventions, and *the observed 
beha^rilDr ~p"a tt'e~rns~o^ "regul'a^t^^ — 

The various norms, rules, laws, procedural""^cbnven- 
tions, and observed behavioral patterns of 
regulatory agents that constitute the normative 
structure of the KPU system. 

An agent responsible for establishing, changing, 
or implementing one regulator or a set of regu- 
lators. 
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SUMMARY 



This project is one of several predesign studies commissioned by the 
National Institute of Education's R&D System Support Program to help the 
Institute establish design requirements for an external monitoring system 
it has proposed to develop. The pursuit of such a monitoring capability 
is in direct response to NIE's congressionally bestowed mission to "help 
build an effective R&D system" as well as its own recognition that "there 
is a great need for better data concerning the knowledge production and 
utilization system and the operating school system it serves .... 
Generally we have lacked both the data base and the understanding of 
system dynamics needed for effective, rational policy-making." This use 
of the term knowledge production and utilization (KPU) stems from the 
Institute's recognition that research and development is a misleadingly 
narrow term to apply to improvement-oriented change activities in education 
and that a more encompassing conception is needed to adequately legitimize 
the full range of activities necessary to help solve or to alleviate the 
problems of American education. 

The central objectives of this study were to investigate the "norma- 
tive structure" (the governance system) of KPU; develop an analytical 
framework through which to understand how formal policy acts as a "regu- 
lator" of activities in KPU; describe the major policies of significance 
to KPU and how they influence the governance of KPU processes in ten 
different case studies selected to be widely representative of KPU in 
education; and make\ecommendations that would help in the design of a 
monitoring program. As a whole, the effort was conceived of as an 
exploratory attempt to develop and test the feasibility of a systems 
mapping approach believed to be compatible with the concepts that NIE 
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proposed for use in its monitoring program. As with many systems studies, 
the study was designed in a recursive fashion, where the results of an 
initial period of inquiry were assessed and the study design was reformu- 
lated before proceeding further. 



In our first approach, we sought to: 

• Construct an extensive taxonomy through which all major 
types of agents, policies, flows, and several other 
regulatory influences on KPU could be classified and 
assigned a code for indexing purposes- 

• Develop a basic master system map (to provide consis- 
tency and coherence as detailed maps of KFU subsystems 
were developed) showing all major agents and on which 
all major information, product, and resource flows 
could be depicted. 

• Apply a variety of commonly used systems analysis tools 
and techniques in concert with the maps, the taxonomy, 
and other information in such sources as ERIC and the 
Databook being prepared in a parallel study to describe 
KPU phenomena in a way that would integrate four different 
modes of description: 

- Typological--a multidimensional classification of 
agents, policies, flows, and other aspects of KPU in 
education and its governance structure. 

- Graphical--a series of diagrams that reveal the static 
and dynamic relationships of various KPU system 
elements. 

- Nuinerical--time-series and other indicators that 
express the quantified attributes of the system and 
its parts. 

- Textual--verbal descriptions of research findings, 
laws, guidelines, and other information that expresses 
nonquantified and non-imagistic at tributes of the 
system and its parts. 

Because of the emphasis on the role of formal policies which help reg- 
ulate or govern KPU, and on the role that time-series indicators might have 
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in monitoring KFU , the conceptual image or paradigm initially explored 
could be characterized as essentially that of hierarchical systems. 

Regardless of how one might view the desirability of its attributes, 
this approach was found to be unfeasible as a framework for investigation 
and analysis of KPU in education. KPU is a secondary goal for most agents 
and institutions in the KPU infrastructure; hence the involvement of many 
of these agents and institutions is ad hoc or purpose specific. Unless 
one first specifies fairly precisely what it is one wants to know about 
the KPU infrastructure, efforts to map KPU as a system with various sub- 
systems must be done either at such a high level of abstraction that the 
level of detail is inadequate for realistic analysis or at such a high 
level of concrete detail that the portrayal falls down under its own 
weight, given the limitations of available media. Moreover, there is 
not merely one perspective from which to map KPU, but many, each of which 
illuminates a different set of relationships underlying KPU phenomena. 

We therefore revised our basic strategy and relied on our inquiry 
in the ten case topics to guide the development of a framework that could 
be used to describe various aspects of KPU as seen from various perspec- 
tives and for various purposes, rather than to test the feasibility of 
one that would fit any given part of KPU into an overall general map of 
hierarchically ordered classifications.. Policies, agents, resources, 
and activities, in KPU were the four tfasic terms of reference that we used 
as the basic building blocks in this new pursuit. As requested by NIE, 
we developed a finished taxonomy only for formal policies. 

In pursuing this strategy we developed a flexible methodology that 
allows the researcher to describe the interaction of various configura- 
tions of policies, agents, and resources in the shaping of activities 
through which specific acts or processes of knowledge production and 
utilization take place. The methodology allows one to see systemic 
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relationships without forcing one to systematize them in an overly 
simplistic fashion. To do this requires that the purpose of analysis 
at least be tentatively established by the analyst to provide a basis 
on which to make distinctions and draw connections and inferences. (A 
first distinction the analyst makes, for example, is between the focus 
o:E^-hl^ ±ni:eres1:-aTid xts suxroundlngs^^. ^ the 
extent to which relationships involving parts of a given configuration, 
the configuration itself, or the environment of the configuration should 
be explored.) 

From the results of our exploratory study, we judge that NIE's stated 
objectives for its monitoring program cannot feasibly be fulfilled by a 
system based on the social indicators approach. Although this approach 
is particularly suitable for improving understanding of some activities 
in some parts of the KPU infrastructure (e.g., those activities having 
to do with the distribution and control of fiscal resources that are 
highly specified by formal policy), it is particularly unsuitable for 
improving understanding of others (e.g., those activities relating to 
the actual creation and use of new knowledge that are highly discretionary 
in nature) . 

A variety of recommendations for development of the monitoring 
program were inferred from the findings of the project. 

• NIE should consider and include a variety of conceptual 
viewpoints in the design of its monitoring program. 

• The design of the monitoring program should be based, in 
part, on an explicit consideration of such definitive 
issues as: 

- The degree to which NIE's monitoring program will be 
based on any given conceptualization or paradigm as 
opposed to being based on a deliberate or haphazard 
mixture of conceptual approaches. 

xxiv 
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- The degree to which NIE will try to rigorously articulate 
(i.e., codesign and coordinate) its monitoring program 
and its other governance functions . 



- The degree to which new knowledge will be conceptually 
limited to include only that which results from processes 
and/or products of the institutionalized KPU system. 

-The degree to which data needs and data collection ac- 
tivities of other agencies (particularly at the state 
level) will be explicitly considered in the design of 
the monitoring program. 

• The design of a monitoring program should, in large part, 
be based on the types of information about the KPU infra- 
structure that will actually be needed by major policy 
analysis activities anticipated for the next several 
years. Thus a policy analysis information-needs assess- 
ment should be done as an additional "predesign" study. 

• The assessment of likely impacts of the Buckley Amendment 
on KPU is an immediate activity that well-represents a 
type of policy analysis that will increasingly need* infor- 
mation about the infrastructure of KPU in the future and 
should be undertaken both for its own sake and as a way 
of concretely facing various trade-offs in the design of 
the monitoring program. 

• As a first step in the development of time-series indicators 
of balance and continuity of support in KPU (as well as to 
provide information of vital interest to NIE's constituency), 
NIE should prepare cross-tabulations of its disbursements 
in various categories (such as mode of procurement, sub- 
stantive topic, type of performer) for inclusion in subsequent 
editions of its Databook . 

The final report of this study comprises two volumes, the first of 
which describes the conduct of the study as a whole, the methodological 
fram'ework that was developed, and the recommendations that were inferred. 
This voltmie also contains an annotated bibliography of various topics 
covered by the study. The second volume demonstrates, the use of the ; 
framework and describes the subs tantive findings that resulted when the 
framework was applied to the following ten case topics: 
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I National Institute of Education (NIE) Allocation 
Policy 

II Federal Procurement Policy and Knowledge Production 
and Utilization in Education 

III Assessing the Impact of Policies that Control the 
Availability of Information 

IV The Far West Laboratory as a Research and 
Development Performer 

V Minicourses as an Example of Policies Affecting the 
DisseminationAr tilization of a Successful R&D 
Product 

VI The School Mathematics Study Group (SMSG) Project 

as an Example of Policies Affecting the Dissemination/ 
Utilization of an R&D Product 

VII ESEA Title III Teacher Initiated Innovation Program: 

An Example of Policies Interfacing Levels of Government 

VIII The Governance of Knowledge Production and Utilization 

in Intermediate Service Agencies: Boards of Cooperative 
Educational Services in Colorado and New York 

IX Policies Affecting the Results of the Federally 

Sponsored Pilot State Dissemination Program in South 
Carolina: 1970-73 

X An Approach to. Monitoring the Role of Government 

Policies in the Process for Selection and Evaluation 
of New Instructional Materials 
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I THE EFFORT TO MAP THE KPU SYSTEM 



Overview of Section I 

The National Institute of Education has been charged with the re- 
sponsibility of contributing to educational knowledge production and 
utilization (KPU) by (among other things) helping to build an effective 
educational R6cD system . In executing this charge NIE asked Stanford 
Research Institute (SRI) to develop a strategy for describing the pattern 
of the educational research and development (R&D) system that exists and 
for identifying the key regulators of that system. In response to NIE's 
request, SRI attempted to create an analytic framework based largely on 
systems mapping techniques. We drafted a highly aggregated master map of 
the whole educational R&D system to provide consistency and coherence as 
more detailed maps of KPU subsystems were produced. This approach became 
unmanageable^ however, because we found that there is not one single per- 
spective from which to map KPU and most of the components of KPU are first 
and foremost components of other systems. No agreement could be reached 
on what the basic subsystems of KPU really are. Iii fact, 'it became clear 
that KPU is not one single system, except by definition. 

The Emergence of Research and Development 
Systems Thinking in Education 

The concept of KPU was first introduced by Machlup (1962) and re- 
cently promoted as a term of reference in education by NIE (Task Force 
on Resources Planning and Analysis, 1973). KPU is a term meant to substime 
the older term "educational research and development." The objective of 
this project is to develop a method for describing the infrastructure of 
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the KPU system in education. To understand this objective and why it is 
reasonable to think in terms of a KPU ''system*' it is useful to review 
briefly the development of federal involvement in educational R&D.'* 

Research and development conducted at federal expense for elementary 
and secondary education is a recent and still relatively small-scale 
phenomenon. For all intents and purposes we can say that federal encour- 
agement of educational R&D had its birth in the cold war following World 
War II. In particular, Sputnik and the space race brought national atten- 
tion to the problems of improving the schools. Following the R&D program 
model developed for defense and industrial efforts, the Cooperative Re- 
search Act of 1954 and the National Defense Education Act of 1958 provided 
support for similarly structured R&D by the U.S. Office of Education (USOE) 
and the National Science Foundation (NSF) . Under this model, educational 
R&D was mainly oriented toward the development and/or improvement of 
educational materials. 

The first educational R&D strategy adopted by NSF and USOE was that 
of promoting the individual creativity of professional scholars to produce 
and incorporate new knowledge into instructional materials. 

By 1964, it was evident that the original R&D model was seriously 
inadequate because it focused on knowledge production and all but ignored 
the coordination of production activities with the needs and concerns of 
the user school systems. A reformulation of the "basic problem" turned 
attention to the fact that state and local education agency support sys- 
tems did not enable teachers to obtain and use the basic educational R&D 
that was being produced. 



(See Gideonse, 1970, Levien, 1973, and National Institute of Education, 
1973 and 1975 for a more detailed history of federal involvement in 
educational R&D) . 
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Through the' Elementary and Secondary Education Act (ESEA) of 1965 a 
new image of educational R6cD was formalized. The federal government had 
to deliver R&D all the way to the classroom or school; hence the research 
development-dissemination-adoption (RDDA) model was adopted as the ideal. 
Moreover, R6cD was beginning to be pictured as the activity of a network 
of institutions including: 

• Universities and colleges. 

• Independent nonprofit organizations, including the regional 
education laboratories and centers. . 

• Profit-oriented organizations . 

• Linkage agents like ERIC, educational publishing firms, and 
formal or informal networks or associations of educational 
professionals . 

• School systems and state education agencies (SEAs). 

• The schools themselves as the "operating system" for education. 

Along with the growth of interest and support of R&D in education, 
the concept of an RDDA "system" began to arise, due in part to the success 
image garnered by systems analysts in the Department of Defense, in part 
to a feeling among educational researchers with a social engineering 
orientation that educational reform could be more successful if it was 
more systematic, and .in part to other factors. These events set the stage 
for a rhetoric arguing that educational R&D should be viewed holistically 
as a system with nested subsystems, a whole range of actors, a.nd institu- 
tions who (should) perform various, specific KPU activities that (should) 
articulate with each other. Expectations for educational R&D had moved 
into the "systems" phase (see Table 1). Thus, the conceptual expectations 
and institutional components for a public sector educational R&D network 
emerged piecemeal over two decades. Finally in 1972 Congress created NIE 
to coordinate federal contributions to this network. The central policy 
behind NIE's creation and operation was very general; it set forth four 
major concerns f'or the Institute: 
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1. Help to solve or to alleviate the problems of, and 
promote the reform and renewal of, American education. 

2. Advance the practice of education, as ah art, science, 
and profession. 

3. Strengthen the scientific and technological foundation 
of education. 

4. Build an effective education research and development 
system . 



Table 1 

EVOLUTION OF THE FEDERAL IMAGE OF KPU: 1954-1975 



Period 


Perspective of KPU 


1954 to 1962 

1962-1965 
1965-to present 


Development of new knowledge by encouraging through 
support individual professional scholars to focus 
their creativity on improving instructional materials 
and basic methods. 

Development of the basic elements and expectations 
for a KPU network. Emergence of the RDDA concept. 

In keeping with the national shift to program 
governance and systems management/management by 
objective, the emergence of the concept of KPU as a 
social system encompassing improved linkage, feed- 
back, and systems monitoring. 



This brief overview shows how it happened that at its inception NIK 
was charged with the responsibility of viewing educational R&D as a system 
and helping a system mature. Unlike the areas of R&D in defense, aero- 
space, and agriculture, however, the goals of. educational R&D are highly 
amorphous, its participants less easily identified, and its role 



[Education Amendments of 1972, PL 92-318, Section 405 (a) (2)]. 

4 



26 



vis-a-vis its clients less clear. Hence, while setting immediate prior- 
ities and distributing national resources to specific educational R&D 
projects, NIE has continuously struggled to develop both an adequate ges- 
talt of the scattered activities it is to help support and a method of 
determining how, as a whole, the nation's KPU efforts can be coordinated 
for a higher payoff. 

In two papers, "Building Capacity for Knowledge Production and 
Utilization in Education'' (Task Force on Resource Planning and Analysis, 
1973) and "Modelling a National Educational R&D System" (Churchill, 1974), 
NIE inquired how best to identify the anatomy of the KPU system. In 
"Building Capacity," NIE argued that an essential problem afflicting KPU 
is a lack of understanding of the structure and process of the system 
(the infrastructure). It therefore was important to develop a capacity 
for monitoring the structure and process of the R6tD system operating in^ 
the field of education. In "Modelling a National Educational R6tD Sysizem," 
Churchill argued that developing a method of describing or mapping the 
anatomy of the KPU infrastructure ought to be one of the first steps in • 
exploring the feasibility of an educational monitoring program. 

I 

Initial Orientation of Project Staff 

This is a predesign study to develop a methodology for a description 
of the KPU infrastructure that is useful for understanding and for 
locating weaknesses or imbalances in the KPU governance system. 

In preparing a study plan that would be responsive to NIE's requests 
and move one step further toward refining feasible expectations for a KPU 
monitor program, we set four tasks: 

1. Develop a general reporting or analytic framework for 
systematically describing the KPU formal governance 
infrastructure. Include as integral to that task the 
development of an expandable taxonomy of public policy 



categories relevant to KPU (one that could in 
principle reflect various informal regulatory 
mechanisms) • 

2. Select ten major "regulators" (governance mechanisms) 
of KPU and describe them using one common framework.'' 

3. Assess the validity and potential benefit of using 
this framework for description in a monitoring 
program. 

4. Extract a set of recommendations and implications 
for the design of the monitoring program. 

At the behest of NIE we limited the domain of study to present 
public policy regulators (PPRs) in the infrastructure of KPU^ activities 
aimed at grades K through 12. (Table 2 lists the features of PPRs of 
interest to NIE. ) 

The analytic framework was to be a descriptive method to organize 
effectively the collection and summarization of data on KPU as a system. 
The framework was to be based on a systems-oriented view of KPU and 
modified by the empirical properties of KPU as we observed them. In 
this sense the framework would lie between the level of theory and the 
level of discrete empirical variables (see Figure 1). As such, the 
general purpose of the framework is to identify and describe the gover- 
nance infrastructure of KPU--more precisely, the role of formal policy 
in influencing the production, dissemination, and use of new knowledge 
for education. 

The analytic framework is not meant to stand by itself; it is con- 
ceived as a component part of NIE's overall approach to policy analysis. 



Our basic approach was that of hierarchical systems mapping which we 
believed consistent with NIE's proposed social indicators approach. 

t 

A bibliographic essay of general systems theory that summarizes the 
rationale for this approach is presented as Appendix B- 
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Table 2 



A TYPOLOGY OF KPU PUBLIC POLICY REGULATORS 
OF INTEREST TO NIE 



Type of policy ; 
Statutory law 
Case law 

Public regulations issued by governmental bodies 

Federal agency decision structures regarding the allocation and 
commitment of funds 

Federal agency requirements regarding project monitoring, reporting, 
planning, and evaluation ^ 

Internal regulations or organizations involved in the creation, 
production, distribution, or utilization aspects of KPU (e.g., 
regulations of state or local educational agencies regarding adop- 
tion of textbooks or other innovations) 

By-laws, guidelines, or codes of ethics that shape the KPU activi- 
ties and support of professional associations and private 
foundations 

Institutional setting or level : 
Federal 

State and intermediate education agencies 
Local education agencies 

Professional organizations :and labor unions 
Private foundations 

Domains affected by policy : 

Level of educational public for R6eD (where applicable, e.g., 
elementary school) 

Content area (e.g., curriculum development, educational television) 

Position on the KPU spectrum (research, development, linkage, 
utilization) 
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Project Methodology: Phase 1 



To meet the objectives and perform the tasks outlined in the pre- 
ceding discussion, we designed the study approach shown in Figure 2." 
We adopted the prevailing hypothesis that KPU is a system bf research, 
development, linkage, and utilization as our initial conceptual orienta 
tion. The preliminary set of categories suggested in Table 2 were used 
to provide an orientation for the taxonomy of public policies. 

From this perspective, we began our preliminary survey of the lit- 
erature and interviewed a variety of actors in KPU to learn their 
perspectives and in particular to get their nominations for the most 
significant issues, types of policies, and KPU agents. We list below 
the most typical KPU regulators that turned up in our survey. 

1. Money (amount of material and fiscal resources) 

2. Stability of support 

3. Concern for human subjects 

a) Regulations on the treatment of human subjects 

b) Militance of subjects and their representative 
organizations 

c) Minority rights 

4. Styles of organization and administration of KPU 
performers 

5. Clearance procedures 

6. Publishers aversion to risk . ' 



The underlying assumption of our approach was that KPU is a system rea- 
sonably characterized by the logical pattern of research- development- 
linkage-utilization. Therefore, the metalogic of the project was that, 
by clarifying a systems description of KPU, we would have the common 
frame of reference for underpinning both a description of any sector of 
that system and the development of categories and classification rules 
for a taxonomy of PPRs of KPU. 
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7. Weak incentives for basic education research 

8- "Politics" (especially of KPU programs at the federal 
level) 

From this survey we would complete the basic structure of the analytic 
framework and simultaneously develop survey protocols for interviewing 
KPU actors throughout the system. We would also develop preliminary maps 
of the infrastructure surrounding specific regulators and would make 
informed decisions to limit the scope of parameters that should be studied 
in the next two tasks (see Figure 2). In turn the results of these later 
tasks would be fed back into the basic analytic framework, particularly 
at the interim report, and necessary incremental adjustments to the 
framework and taxonomy would be made. 

In the process of conducting the survey, we were able to further 
clarify what the analytic framework should be. Based on Figure 1 and 
our expectation of what tools would be useful to describe the range of 
infrastructure of KPU, we expected the analytic framework to have two 
principal features. 

1- It should rest on an underpinning of a complete and 
consistent image of KPU that would allow the relations 
and connections of the parts of the system to be mapped 
and missing or overlapping connections to be identified. 

2- It should provide a complete and consistent set of 
taxonomic principles to allow classification of the 
data relevant to the regulation of the KPU infra- 
structure. 

However, because we encountered more ambiguity than anticipated in 
forming an overall picture of KPU, development of protocols and selection 
of topics proceeded at this stage without the coordination we anticipated 
the framework would provide - 
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Conference with Advisory Panel 



At this point, we held a design conference with the advisory panel. 
Two related issues were of central concern. First, if the analytic 
framework was to serve as the set of principles with which to underpin 
that part of a monitor program that deals with the dynamic regulation of 
the KPU infrastructure, then what features must the framework have? 
Second, how might we proceed to formulate such a framework? 

We decided that: 

1. The analytic framework should be capable of illuminating 
the entire KPU infrastructure in education. 

2. The KPU "system" should be mapped in such a way as to 
reveal potential linkages that are missing and needed 
as well as existing linkages. This is particularly 
important for operational linkages between knowledge 
production and knowledge utilization. 

3. Policies, agents, and the flow of information and 
resources should be the primary categories from which 
to select topics to test the analytic framework. 
Topics should be selected to give as balanced a 
coverage of the spectrum of KPU activities as is 
feasible. > 

The problem of how to develop the analytic framework did not marshal 1 
a set of recommendations as specific as the expectations listed above. 
However, a list of somewhat eclectic orienting questions was produced, 
to which the framework should provide specific answers: 

1. What policies influence most greatly the actions of 
various KPU agents and the aggregate actions of various 
KPU subsystems? 

2. What policies tend to be ignored or overridden in 
practice? 

3. What if any agents or areas of the* KPU system in 
education suffer most from "policy overload" to the 
point of becoming dysfunctional? 
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4. Do any of the effects in (2) and (3) seem to occur across 
KPU subsystems or agents of various types (i.e., are 
there particular policies that seem especially significant 
or insignificant in terms of their actual influence 
throughout the system)? 

5. Does the length of the timing cycle of evaluation and 
feedback make much difference in the effectiveness of 
a policy? 

6. What recent changes in KPU policies have been made that 
influenced the behavior of KPU agents? What are the ef- 
fects of such changes? 

7. Are there significant omissions of particular policies 
that are needed as a result of recently changing 
context? 

8. Are there in effect adequate second-order regulators 
(that is, regulators who control the development and 
persistence of other regulators, for example, educa- 
tion subcommittees in Congress)? 

9. What policies will be either necessary or likely because 
of expected changes in the context of KPU in education? 

The framework should also provide assistance in satisfying secondary 
objectives for the overall study by (1) indicating the kinds of informa- 
tion that should be collected on a regular or periodic basis as a part 
of a program for monitoring KPU, (2) providing implications for the design 
criteria for a monitoring system, and (3) serving as a basis for identify- 
ing special research studies to understand the operation and effects of 
specific policies or groups of policies in general and on specific sec- 
tors of KPU. ^ 

As a result of this first stage of effort and our meeting with the 
advisory panel s.everal specific issues became evident- First, both the 
current literature and prevailing sentiment of the advisory panel pointed 
to the lower levels of the operating education system as the sector in 
terms of which KPU needs to be understood. Attention should be paid to 
KPU agents, actors, and regulators in the local education agencies (LEAs) , 
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school sites, and classrooms. Second, KPU activities can only be ex- 
plained in terms of an adequate overall picture of the KPU network and 
its basic dynamics. Third, because a complex endeavor like KPU is 
controlled by a variety of influences, the question, ' How influential 
are formal public policies? must be asked. 

Each of these issues had a profound effect on how we performed the 
cycle through the three tasks. The LEA issue led us to focus the pre- 
liminary field work in that direction. LEA staff (particularly those 
engaged in installation of new practices) were the principal agents 
interviewed. The issue of building an analytic framework for describing 
the KPU infrastructure on the foundation of a sound picture of the KPU 
network led us to attempt to create an overall or master map of the basic 
subsystems of KPU to allow unambiguous identification and classification 
of KPU agents, activities, and policies. Finally, the issue of formal 
regulators of KPU led us to attempt to map the specific dynamic inter- 
actions in the KPU infrastructure and to identify and describe the most 
significant regulators of that interactive process using systems dynamics 
methods. 

With this orientation the analytic framework would be the basic 
master map, the principal subsystem maps, and a set of tools for identify 
ing the key elements of the map, expanding the maps, and describing or 
tracing the effects of changes of^variables or parameters in the maps. 

Developing a Concept of Public Policy Regulators 
of KPU 

The reasonalDleness of attempting to create a specific overall map 
of KPU as a" system arises from the assumption that there are in fact 
distinct regulators of the KPU infrastructure in a cybernetic sense, 
that is, specific regulatory processes or functions. Therefore at. this 
point in our effort we attempted to clarify our concept of PPRs. 
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We expected that, whatever "regulators" are, the types of regulators 
probably vary sharply according to whatever goes through the KPU pipeline. 
However, behind each specific formal regulator we could expect to find 
one or more people acting as the agents who do the regulating. (This 
meant that we were looking at neither informal regulators nor highly 
diffuse public regulators). Gradually we put together a profile of PPRs. 

Profile of Public Policy Regulators 

Because of the proliferation of PPRs it is important to concentrate 
on only those called out specifically for study in specific cases. We 
must find a way to wash out or disregard those PPRs not immediately 
significant and simply note that they are contextual PPRs existing as 
part of the system but not needing to be described in detail to under- 
stand the point at hand. The remaining PPRs would be those that are 
significant to the system of description. 

Significant PPRs will be called out by whoever is responsible for 
each study topic. Having located a significant PPR, the contextual PPRs * 
should be identified and quickly examined to see if there are other sig- 
nificant PPRs or whether several contextual PPRs taken together become 
significant. The contextual PPRs should then be worked up on a contextual 
PPR worksheet and the significant PPRs should be worked up on significant 
PPR worksheet. 

Having identified and analyzed the PPRs for a given study topic or 
area of concern, the contextual PPRs may be given final mention and left 
to rest. Significant PPRs, however, might be treated in the same way as 
agents on the systems map. By placing each significant PPR at its place 
of origin on the map (by an overlay) and indicating those areas which it 
regulated on that map, such things as PPR "overload," "conflicting" PPRs, 
and the like, should be readily identifiable. 
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Developing a Holistic Image of KPU 

The scope of the project was now limited solely to developing a 
strategy for describing the formal regulatory structure and dynamics 
controlling KPU. Therefore we assumed we would be able to map this 
structure without too much difficulty. We began a search of the education 
literature for some portrayal of the various tiers of government and how 
they interact to provide governance of education. Presumably such infor- 
mation would allow us to develop a master map of KPU in a straightforward 
way. 

All we could find in our search, however, were general treatises on 
the theory of governance (e.g., Lindbloom 1968) and the specific network 
of actors that converged around specific legislation (e.g., Summerfield 
1974). We took this as but one more indication of the immature ^tate of 
the art that deals with the systems of educational governance. There- 
fore we proceeded to construct our own general description of this network. 

We identified concepts for inclusion in the general model to underpin 
the descriptions. First, KPU is clearly composed of functional subsystems , 
that is, networks of agents engaged in a particular activity within the 
wider range of KPU. For example, curriculum development is a functional 
subsystem. Second, KPU is understood most commonly in terms of various 
conceptual subsystems , that is, organizing classifications of the major 
KPU sectors. For example, at the extremes of the spectrum, are the 
knowledge production and practice support conceptual subjsystems. Third, 
the formal regulators of KPU are clustered by the legal profession and 
cultural tradition into various legal subsystems . For example, federal 
law and state education codes are legal subsystems. 



In fact this indicates a much more fundamental problem, as we will show 
later in the report. 
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Related to how subsystems should be conceptualized, we found three 
additional considerations needing to be included in the framework. First, 
the connection or interface of social subsystems is, we found, effected 
through people and many of these human interfaces do not follow the main 
line of authority in the Weberian sense. The analytic framework must be 
able to be used to get down to that level of detail where the interface 
between subsystems is actually conducted by specific people especially 
when those people are at the lower levels of the organization chart 
operating in channels not formally designated or controlled. Second, 
there are likely to be "big engines" driving change through the system, 
such as the Sputnik-science drive and the civil rights movement. The 
analytic framework should be able to portray impact of these factors on 
the KPU infrastructure. Third, we should not expect to find all the 
necessary and desirable subsystem connections already existing. The 
framework should reflect the extent to which there are subsystems that 
are not well interconnected in KPU. 

With these considerations in mind our problem is to create a framework 
involving an overall image of KPU that will allow us to identify the rules 
that govern the KPU infrastructure. 

The first step in the development of an analytic framework centered 
on identifying all types of agents in the KPU system and noting their 
roles vis-a-vis two basic graphics. In one .of these graphics KPU agents 
are sketched hierarchically according to the functional paths of informa- 
tion, authority, and the like that connect them (see Figure 3a). The 
numbers are keyed to an exhaustive table that specifies the tentative 
paths- and generic sources of policy information marking them (see Table 3). 
Figure 3a is our master functional subsystems map. 

The second map. Figure 3b, identifies the orientation of each agent 
with respect to KPU conceptual subsystems. The nui)ibers on this map des- 
ignate agents and are keyed to Table 3b. From the exercise indicated 
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Table 3 

TABLES IN SUPPORT OF HGURE 3 



{a) Regulation Matrix (PPRs Shown on figure 3a) 



To 



Source of PPR Authority 



Control Issue 



1, Department AdminlHtratlun 



Department AdtnlnliStrat ion 



Teacher desirous of 
using extra learning 
environment 



State Education. Code 

Local District Teacher's Manual 

School Site Regulation 

State Education Code 

Local District Teacher's Manual 

School Site Regulation 

Admin Is t r(i t Ive structure ; 
State Education Code 
Local District Teacher's Manual 
School Site Regulation 



Activity of student 



Conduct of classroom 



Use of environment 



(Complete Matrix Shown In Interim Report (Markley, 1974) 



CuuHress 



Congress 



34. 



Courts (all levels) 



State legls latures 



Materia Is producing 
Inst I cut Ions ; research 
and testing Institutions 

Agents chosen by 
litigants 



U.S. Constitution 



U.S. Constitution 
20 U.S.C. I23a 



Congress 
Executive 



U.S. Consltuclon 
State constitutions 



National Education 
Policy Goal Setting; 
confirmation of 
nominated agency 
heads 

Limitation on 
Federal Involvement 
In education 

Conduct of surveys 
"0MB clearance" 



Adjudication 



(b) Agent List (Numbers Shown on Figure 3b) 



Number 


Agent Name 




Number 


A^cnt Name 


1 


Foundation program In Education (c.{ 


J. , I/D/E/A) 


14 


Intermediate Service Agency (e.g., Teacher Center) 


2 


Federal Agency Program Office (e.g. 


NIE) 


15 


SDE specialists 


3 


Federal Agency Policy Office 
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SDE Planning Office 


4 


SDE Research Office 




17 


Regional Information Centet' 


5 


Policy Research Agency 




18 


Publishers 


6 


Bislc Research Agency 




19 


Accrediting agencies 


7 


Mission /Applied Reiearch Agency 




20 


Testing and evaluation research agencies (all types) 


8 


Development agency 




21 


Professional Associations 


9 


Professional Journals 




22 


Local administration 


10 


Institutions 




23 


Principal 


11 


Personnel training 




24 


Student 


12 


Problem-solving consultanti 




25 


Learning Center 


13 


In format ion Storage /Retrieval /Dissemination 
Agency (e,g, , ERIC) 


26 


Teacher 
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in Figure 3b we hoped to gain a purpose-oriented perspective of KPU to 
understand the hierarchic authority-based approach in Figure 3a, 

With these two global maps drafted, we then turned our attention to 
fleshing them out via study of the individual agents and policies. Figure 
3c shows how we expected to be able to do a more detailed description of 
any KPU agent using the agent as the focus of the display. 

We assumed that we .would spend the duration of the project elaborating 
the Figures 3a and b and developing maps like Figure 3c for the relevant 
regulators chosen as case study topics. 

The Taxonomy of Elements and Relations in 
the KPU Infrastructure 

The importance of an adequate taxonomy of elements and relations 
became clear as soon as the first maps began to take shape. There were 
hundreds of components to keep track of and thousands of possibly rele- 
vant interrelationships. If the collection of influences that have im- 
pact on the teacher^ for example, are to be easily identified and tracked 
and the second-order regulators of those influences are to be identified 
in turn, then one common classif icatory structure that could serve as the 
basis for a data storage and retrieval system had to be designed. 

The taxonomy had to meet three criteria. First, it had to 
provide space for classifying all the different phenomena making up the 
KPU inf rastniction (e.g., each type of agent, policy, resource, and 
relationship). Second, it had to be useful for coding the connection 
between each element and the rest of the system. Third, it had to be 
useful for retrieving all the significant connections and elements that 
surround a given element (e.g., see Figure 3c). 
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Our initial research indicated a variety of considerations that 
must go into the design of a taxonomy to meet these goals. The KPU system 
is generally considered to consist of a number of agents who control, are 
controlled by, and process the various flows of influence relevant to KPU 
in education. Agents regulate KPU activities by making, administrating, 
and responding to various policies. Policies dictate both structure and 
process in the system. Formal policies are codified and have some legal 
basis. Timing considerations are considered important because most 
policies include requirements for certain things to be done at certain 
times; therefore, a set of timing factors is included. Table 4 shows a 
sample of the first taxonomy designed to be responsive to these expec- 
tations. 

Based on this first section of the taxonomy, we drew up a gen- 
eric agent list (see Ta.ble 5). The list allowed for additions indefinitely 
as needed and the assignment of a set of location and function identifica- 
tions to each entry, based on the functional and conceptual subsystem maps. 

Our goal was to develop similar generic tables in the other 
categories of the taxonomy, such as flows, policies, informal regulators, 
timing, and operating conditions. These tables were not yet developed at 
the time of the interim report. 

After solving the problem of providing room in the taxonoitiy to 
expand it as needed and to show how each component fits into* the larger 
picture, we addressed the problem of cross-referencing each component with 
all the relevant connections that converge on it and which it influences 
by gathering all the data around each component into its own table. This - 
(input-output) table was constructed to complement a graphic. For example, 
Table 6 is a compilation of the major components of the KPU system- that- 
converge on the classroom teacher . (Table 6 is thus complementary to 
Figure 3c. ) 
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Table 4 



TAXONOm OF ELEMENTS AND RELATIONS IN THE KPU INFRASTRUCTURE 



Class/SubcIass 


Tentative 




Class/SubcIass Tentative 


Identification 


Code 




Identification 


Code 


Agent function (I-A) 






Informal 




Knowledge production 


AFin 




Knowledge Utilization 


AF40,..,. 


Support 






Adminis tra tive 




Research/development 


AF,2 

i. ^ 




Teacher/facilitator. 


AF42 


Review/evaluation 


AF,^ 




Student /learner 


AF43 


Regulation 






Political influence 




Initiative 


AF21 




Agent location (I-B) 


AL 

nm 


Policy governance/ 
guidelines 


AF22 




Federal 
State 


AL20 


Fiscal control 
Technical control 


AF23 
AF24 




Regional/county 
District 


AI.30 

ALcQ 

A T ^ 


Litigation 


AF25 




School 


Professional standards 


^26 




Learning setting 


Linkage 


AF30 




Voluntary and 




KPU information linkers 


AF31 




miscellaneous groups 


AL70 


Consulting 


AF32 




Philanthropic foundations 


AL71 


Personnel training 


AF33 




College/university 


AL72 


Professional media 


AF34 




Nonprofit corporations 


AL73 


Mass media 


^35 
AF36 




Profit corporations 


AI.74 


Material marketing 




Associations 
(including consortia) 

Ad hoc groups 


AL75 

AI«76 


Note: Within a major class. 


agents are 


described by a two-dimensional matrix. 


with . 


I-A and I-B providing 


the respective axes. This project team, insofar 


as it 


is a component of SRI, 


would be located in cell AF -j^ : ALy^ . 
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Table 5 
GENERIC AGENT LIST 



Numeric 
Code'' 


Agent Name 


Location 
(AL)*'' 


Function 
(AF) 


1 


Federal Educational Agency (includes No. 
2-10 below) 


10 


20, also 

those 

below 


2 


U.S. Executive (President, Executive 
Office) 


10 


22 


3 


U.S. Congress 


10' 


21 


4 


Office of Management and Budget (0MB) 


10 


23 


5 


Department of Health, Education, and 
Welfare (DREW) 


10 


20 


6 


As si s tant Secre tary for Education (ASE o f 
DHEW) 


10 


20 


7 


National Center for Education Statistics 
(NCES) 


10 


11, 12 


8 


U.S. Office of Education (USOE or OE) 


10 


20 


9 


National Institute of Education (NIE) 


10 


20 


10 


Formally Established EKPU Advisory Groups 
(Federal) 


10 


32 


11 


Federal courts 


10 


25 


12 


Other Federal agencies having EKPU- 
related functions 


10 


20 


13 


Advocacy groups (federal level) 


70 


50 


1 L 


o Ld Lc rja,UL.ci Lxu n rigency 




C\J 


15 


State Executive Officer (Governor) 


20 


22 


16 


State Legislature 


~20 


21 


17 


State Board of Education (SBE) 


20 


22 


18 


Chief State School Officer (CSSO) 


20 


21, 22 


19 


State Department of Education (USOE qjev- 
DOE) :^ 


20 


20 
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Table 5 (Continued) 



Numeric 




Location 


Function 
(AF) 


20 


Formally established EKPU advisory groups 


20 


32 




91 


ucner scace agencies naving r4iN.irU"reiaLea 
functions 


9n 


20 




9 9 


iiuvocacy groups ^scace LeveLy 


/ u 


50 




23 


Research and testing institutions 
V^mciuamg kocu^ 


70 


10 




24 


State courts 


20 


25 




25 


Material producing institutions (includes 
publishers) 


70 


36, 


50 


26 


Personnel training institutions (Teachers 
Colleges) 


70 


33 




27 


Professional associations and consortia 


75 


26, 31, 
50, 

others 




Professional media 


/ u 


35 






Academic sector (colleges and 
universities) 


79 


10, 


30 


on 


Regional Education Agency 




30 




31 


Local Education Agency (LEA) 


40 


20, 
41 


30, 


32 


County Department of Education 


30 


20, 


30 


33 


Local Board of Education (LBE) 


40 


21, 

23 


22, 


34 


School District Administration 


40 


22, 
24, 
41 


23, 
31, 


35 


School Administration 


50 


22, 
24, 

41 


23 
31, 


36 


Department Administration 


50 


26, 


31 



ERIC 



24 

46 
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Table 5 (Concluded) 



Numeric 
Code'^ 


Agent Name 


Location 
(AL) + 


Function 
(AF) 


37 


Teachers (learning setting, classroom 
environment, and so on) 


60 


31, 42 


38 


Students 


60 


43 


39 


Formally established EKPU advisory groups 
(local) 


30, 40, 
50 


24, 32 
50 


40 


Advocacy groups (local level) 


70, 76 


50 


41 


Local courts 


30 


25 


42 


Teacher centers 


30, 40, 
71, 72, 
73, 74 


31, 33 


43 


Extra classroom learning environment 


60 ■ 


37 


44 


Electorate 


70 


50 


45 


Attorneys 


70 


25 



Note: Additional agents, a lower level of aggregation, can be 

identified according to the same protocol; for example, a school 
district curriculum specialist would be AL(40), AF(31) and a mem- 
ber of Agent No. 34. 

A code for purposes of abbreviation on the maps and charts, 
t 

Partial listings only. 
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Finally, by taking the information thus gathered, we planned to 
create a series of overlays for the basic maps to show how influences 
converged on specific agents through "process loops." Figure 4 shows the 
basic template of LEA agents and the six overlays applied to that template. 

The Image of KPU 

At this stage in our project the field investigations, aimed at 
providing empirical support for our image of KPU, proceeded fairly straight- 
forwardly. They seemed to be providing the incremental refinements for the 
analytic framework we expected they would. 

We began to realize that KPU has more governance levels than our 
first functional subsystem map portrays. At the minimum, most states have 
regional or countywide educational service agencies which hold a formal 
governance" position in KPU halfway between the LEA and the state. This 
would necessitate adding more hierarchic levels to the functional sub- 
system map and expanding the sophistication of the linkage categories in 
the taxonomy. We still saw KPU as having a regulatory structure marked 
by formal policies, but we were surprised to find both the sheer volume 
of policies and yet the amount of discretion many KPU agents have espe- 
cially at the LEA level. Having expected to find real conflict among regu- 
lators and instances of policy overload^ we interpreted the myriad stories 
we heard of thwarted effort as evidence that this search would be fruitful. 
While we still expected to find that KPU is a system, we found evidence 
of the suggestions we had read about in the literature, namely., that KPU's 
formal sectors seem to be clustered into two almost distinct subsystems, 
a KP subsystem and a KU subsystem. 

Several important but small clarifications of basic concepts 
began to take place. We found that PPR is an ambiguous term that refers 
to some sort of disembodied dynamism. Formal policies made and used by 
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agents to guide activities are what mark this dynamism and make it moni- 
torable. Similarly, flows are monitorable only in terms of the resources 
and information that are exchanged between agents. 

The most serious difficulty seemed to be in the area of deter- 
mining the significance of KPU system components. Except in terms of the 
gestalt of the case topic writers, there seemed to be no way to get a 
frame of reference and a standard to determine significance. Even the 
master systems maps were little help without a preconceived notion of 
what was valuable and what were the cause and effect links. However, we 
assumed that this problem would gradually be resolved with the elaboration 
and refinement of the analytic framework. As the general maps became more 
detailed and the context more clear, it would be possible to recognize 
"significance" with greater ease. (We did not consider the possibility 
that it might get more difficult as the conflicting values of more con- 
stituencies were considered.) 

Project Status at the Interim Report 

At the end of the first six months of the project as we prepared 
the interim report, we had a basic image of the analytic framework. It 
is a loosely structured set of mapping and taxonomic tools that enable 
better understanding of how the KPU system operates and is governed. A 
necessary first task in the development of the framework entails a pre- 
liminary mapping of significant portions of the KPU system because as a 
methodology the framework is holistically oriented. These maps would be 
the principal guides to what data to monitor. With the understandings 
thus obtained, inferences for both the final formulation of the analytical 
framework and the design of the monitoring system would be made. 
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II THE EFFORT TO DESCRIBE THE KPU INFRASTRUCTURE 

Overview of Section II 

In this section we describe how the basic assumptions and study 
methodology were revised. Rather than attempting to map KPU into some 
classification scheme, we had to alter the whole study strategy. We 
decided that the development of the analytic framework must be empirically 
designed from the bottom up--that is, the description technique must be 
suited to the specific realities of the subject matter on which it is to 
be used. Thus we developed a methodology for describing units of KPU 
without first requiring agreement on an overall picture of KPU. This 
methodology has as its objective the development of alternative descrip- 
tions of the KPU infrastructure as seen from various perspectives and for 
various purposes. The methodology itself is systematic and rational, like 
the scientific method, but it does not presume that KPU is a system or 
that it is complete, consistent, and follows immutable laws. (In fact 
KPU is none of these.) Like th^ scientific method, the analytic framework 
is designed not only to survive but to support radical revisions in our 
theories of KPU. 

By drawing on a series of empirical case studies, we generated a 
preliminary image of KPU. From it we derived specific expectations for 
an analytic framework to guide description of the KPU infrastructure. 



Review of the Progress of the Project: Problems 
with the Analytic Framework and Taxonomy 

Soon after the interim report we gathered together the staff and 
advisors to try to identify , organize , and interpret the problems turned 
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up by the writing of the report. This would allow us to make necessary 
modifications in our concept of the project goal (e.g., what describing 
the KPU infrastructure should mean) and in the methodology of our study 
plan. 

Because the expectation had been that there would be a reasonably 
complete analytic framework and taxonomy at the time of the interim 
report, the fact that there were still distinct problems with each stood 
out as our foremost concern. While the expectation for the analytic 
framework master maps had crystallised, we did not seem to be able to find 
the formula to fill that expectation. The question therefore was, Why? 
Likewise the taxonomy seemed to call for straightforward elaboration of 
categories and generic tables but ambiguity increased as we became more 
concrete, specific, and detailed. What were the sources of the difficul- 
ties and how could they be overcome? 

Problems with the Analytic Framework 

Two components were central to the framework: 

1. A complete and coherent image of KPU on which to base 
a master map (template) of the whole KPU system. 

2. A set of classif icatory and graphics tools to describe 
the elements and dynamics of that system particularly 
in terms of basic functional and conceptual subsystem 
categories. 

The central problem was that we had not satisfactorily clarified 
the first component. Specifically we had not been able either to clarify 
or integrate the basic image of KPU. When we went to the field to 
elaborate the functional master map (Figure 3a) with empirical findings 
we discovered that the more specific the setting the more unique was the 
way in which functional tasks were divided and assigned. Generalizations 
rapidly faded. Thus we were forced either to make the number of categories 
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explode or to assign arbitrarily agents to categories which were not really 
appropriate. Moreover there was a second dimension to the problem. When 
we attempted to pull together the picture of the dynamics of any given KPU 
situation the maps mixed multiple perspectives like cubist paintings. As 
the interim report shows, we resorted to overlays to sort out some of the 
complexity and make reasonably intelligible maps (see Figure 6). However, 
this disguises the problem of picturing the same situation from the mul- 
tiple, contrasting perspectives which ought to be brought to it. 

The search to explain why these problems arose and what to do about 
them at first led us to propose that it was simply the complexity of the 
KPU field. Even with the limits of scope which we had established, the 
proliferation of agents, policies, and activities was enormous. Could it 
be that the descriptive task was simply too large to be manageable with 
our limited resources? Perhaps to describe KPU adequately would take an 
effort similar to that necessary to produce the descriptive analytic 
framework in zoology. 

We reasoned that it ought to be possible to complete a sample descrip- 
tion by limiting the universe to one small school site, for example, and 
by simply describing that. However, the "cubist" maps problem precluded 
even this. There was something more f undamentallly wrong with the approach 
Somehow, through the assumptions and limitations we had placed on ourselves 
we had boxed ourselves into a corner. 

Our experience in conducting the field interviews prevailed at this 
point. We realized that there isn*t one way to conceptualize KPU but many, 
depending on one*s perspectives, values, and reasons for creating an image 
of KPU. The basic approach that we had been taking was a social indicators 
approach. Our hypothesis had been that if we developed one general and 
adequate model of the KPU system we could locate not more than several 
dozen key variables or indicators that could be monitored to track the 
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health and progress of American KPU. But people with different roles, 
values, and political goals will see different aspects of KPU as signifi- 
cant, different relations as binding regulators, and different outcomes 
as v^aluable. 

Moreover we began to realize that there were several dangers in 
trying to create one image of KPU, The approach we were taking at best 
would produce a relatively simple abstract analog model of KPU and not 
actually describe the way KPU itself works. This would leave us with 
very few clues about how to change or mold the actual, KPU infrastructure 
that exists in the izorld. Second, the approach tended to encourage 
reification of one's conceptual categories. We began to try to fit KPU 
events to our classification system rather than recalling that these were 
merely concepts to be used whenever they proved effective. 

Simultaneously, as we delved deeper into the systems literature we 
were struck by the fact that living systems survive because they possess 
sufficient complexity to perceive and respond to the variety in their 
environment, not because they impose highly simplified images on reality ' 
or (worse yet) try to keep the whole of their environment from changing. 

'—Put simply, we had been trying to get the KPU environment to behave 
as if the conceptual subsystems of knowledge production, knowledge utili- 
zation, knowledge production support, knowledge utilization support and 
linkage, or some permutation thereof were the one whole coherent picture 
of KPU rather than designing a way to produce pictures to fit what was 
really happening in KPU. 

Problems with the Taxonomy 

The same explanation also accounted for the difficulty we were having 
with the taxonomy. . 
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Our initial effort toward building a taxonomy of policies centered ^ 
on developing a strategy by which the research team could input to a 
central record (for future cross-referenced retrieval) policies which 
either directly govern or impact KPU that they found through work on the 
case studies. While we had tried to provide flexibility by' leaving room 
to expand major categories and generic lists we were turning up items that 
belonged either simultaneously to all of the major categories or ambigu- 
ously to one or more. Categories often contained most of their items at 
intersections with other categories. Worst of all, we had not successfully 
developed a defensible boundary rule to distinguish what is and is not KPU. 

The difficulty seemed to lie in trying to create one universal tax- 
onomy for a subject matter more properly treated from multiple perspectives. 
Individual writers reported they could recognize and assign significance to 
agents and policies relative to the theme they were exploring but not rela- 
tive to some set of arbitrary universal categories. 

These findings pointed out fundamental considerations that must be 
taken into account in research effort; namely, that it is the particular 
situation in which a policy is applied and the combination of various 
agents, activities, resources, and other policies that determine the sig- 
nificant features of a policy. Hence there is not one set of policy 
categories that will portray the governance of a particular subsystem from 
all the relevant perspectives. The significance of the policy to the study 
being conducted is not always isomorphic with the overall thrust and in- 
terpretation of the policy or its significance to other areas .of KPU. 
Consider, for exi^mple, the study of instructional materials adoption 
process (Case Study X). The researcher is quite justified in finding the 
Ninth and Tenth Amendments to the U . S. Cons titution as significant to the v 
instructional materials adoption process because they have been interpreted 
to preclude the federal government from making direct policy mandates 
concerning, curricula in the public schools. However, to use this property 



of significance to build toward a general policy file would be misleading 

because these amendments are much broader in scope. Yet to taxonomize 

them in their broader sense would both overburden the researcher and 

obfuscate the significance of the policy's relationship to KPU. 

As a result^ while attempting to reformulate pur basic concept of 
the analytic framework and the structure of our study plan^ we shifted 
the focus of the taxonomy away from categorization toward description. 
That is, instead of assigning policies to preset categories, we assigned 
descriptors to policies. Taking our lead from the ERIC filing system, we 
began to experiment with using the taxonomy purely as an indexing system. 
This would allow us to add flexibility to the classification system as 
needed even if it cut across previous categories*. Writers could classify 
elements based on their significance in the context in which they found it 
and this classification could ba expanded as more connections were uncovered. 

While we still retained such major categories of descriptors as source, 
type, and content or target, this shift marked the first major change in the 
direction of our thinking to relative rather than absolute descriptions. 

Basic Assumptions Challenged 

Through this examination of the analytic framework and taxonomy, we 
came to the conclusion that we had become caught up in our own myth and 
rhetoric about KPU and that what we had most to do was reexamine and 
reformulate our assumptions based on observed empirical features of KPTjT^ 

We decided that our basic problem lay in the assumptions we were 
using in an analytic framework. These assumptions in condensed form were: 
as follows: 

1. We had been looking for a universal perspective on KPU when 
education is a complex, value-laden matter that is viewed 
from many contradictory perspectives by different policy- 
setting constituencies. The framework has to be useful 
to enlighten each of these. 
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2. KPU , if it is a system at all, is a social system (i.e., 
artificial, man-made). Hence it can be expected to have 
gaps, inconsistencies, and sources of indeterminacy not 
found in natural systems. Unlike the chemist or astronomer 
we cannot assume that what we are studying is a coherent 
system that will admit of "discipline'* study. At best, 
KPU should be regarded as an eclectic field of activity 
until demonstrated otherwise. 

3. The expectations of finding specific indicators of KPU 
performance, and, moreover, that they should be marked 
by formal policy are probably quite naive. In the first 
place, control or regulation in complex systems is rela- 
tively well dispersed. Management cannot assume direct 
autocratic control of everything that happens. The 
systems concerned are just too big. KPU is a self- 
organizing system which metabolizes the inputs from all 
its components at its many levels and entry points. It 
may be that there simply is no set of specifically identi- 
fiable managers of KPU, yet it works and progresses. 

We had been led astray in studying the nature of the extant and 
accepted managerial process itself. To make studies of the regulators was 
the obvious thing to do; and yet *'the regulator" turns out to be the embod 
ment of something we might call "regulating principle" --that set of 
characteristics discovered in any viable system through which the system 
maintains its organization. 

The regulatory infrastructure is not so much a part of the system 
managed as it is the system's own designer. Because KPU is not a tight- 
knit, well-defined, single-purposed system, decisionmaking is pervasive 
throughout the infrastructure; it takes place at most locations of activi- 
ties and at many different times. Furthermore, when some regulator can be 
identified, it is likely that either the agent or the formal policy is not 
so much the regulator as the pattern of the situation is. 

Several reorienting questions thus emerged. What would happen if we 
shifted the search from identifying the significance of the role of spe- . 
cific actors and policies to the regulatory impact of the patterns of KPU? 
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Instead of testing different versions of a conceptual image of KPU 
to underpin the analytic framework, could we profitably shift the effort 
to develop a. strategy that would expose the structure and role of both 
individual KPU components (policies and agents) and the pattern of com- 
ponents regulating KPU? We began to investigate whether we could move 
back our effort to develop a methodology one conceptual level and design 
a systematic method of inquiry into what the structure of KPU is rather 
than how it can be pictured with one particular image. 

We began to take stock of what we thought were the characteristics 
of KPU, the KPU management system, and what characteristics an analytic 
framework ought therefore to have. We decided that first and foremost a 
paradigm for inquiry into the KPU infrastructure — if it is to be valid-- 
would have to provide a realistic description of the KPU infrastructure. 
This led us to reflect on what our case studies were telling us were the 
important *'real" features of KPU inf rastrucure,* 

In other words, we were taking a reversed approach from phase one 
(a redefinition of "bottom up"). Could we develop a systemic strategy to 
view KPU (like a scientific method), such that the method itself was com- 
plete and consistent but did not presume a holistic concept of KPU as its 
first principle? 

Such a strategy would be a distinct contribution in that it would 
permit development of whole new paradigms of KPU without requiring radical 
revision itself much as the scientific method is only incrementally revised 
when major theories in different fields and disciplines are overthrown. 



By this time we had abandoned our original goal of simply developing an 
analytic framework and then testing it with case studies. We; now were 
increasingly relying on the case studies to provide information with 
which to form and refine the framework. 
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An analytic framework for a monitoring program ought to survive revisions 
of KPU theory (such as R&D becoming RDDA, becoming KPU) without necessi- 
tating radical revision of the methodology itself. Thus the problems 
with the analytic framework led us to consider developing it as a strategy 
for forming "empirical generalizations'' about KPU rather than theories of 
KPU (Kaplan 1964).. • 

Expectations for the Analytic Framework: The New 
Botcom-Up Approach 

Having reached this point of examining the preliminary case studies 
for analytic purposes the framework should meet, we found ourselves 
assembling a list of how the framework could meet these expectations as 
the natural next step of our efforts. Moreover we found we were no longer 
assuming de facto that KPU was a system. The time had come to reconsider 
our study plan directly and reorganize the logic of the remainder of the 
project- 

The list of expectations for the framework we now found reasonable 
follows: 

• The framework must highlight only information that is 
important to a given application. 

• It must cover and reveal the possible impacts of a given 
policy, by identifying all agents, activities, and re- 
sources governed or influenced by a given policy. 

• It must provide a mechanism to identify all policies that 
govern or influence a given agent, activity, and/or 
resource. 

• It should assist in the identification of stages that are 
required by a policy, indicating the specific agents and 
resources involved. 

• It should incorporate ordering into time of activity 
components. 
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• It should display the qualitative difference between 
configurations, for example, those that are ongoing (An 
SEA, a bureau within NIE) and those that are episodic 
(an ad hoc group, such as a certification committee). 

• It should provide a means to link a particular policy 

(or policies) with a particular behavior set, for example, 
asking. For goal-oriented activity A, what policies (x, 
y, z) are relevant? 

• It should provide a means to indicate the evolution or 
causal linkage between various policies. 

An analytic framework should be a set of tools to construct a view 
of the functioning KPU system rather than, as the term implies, that view 
itself. This approach is more reasonable since it allows the researcher 
to take the approach we are developing and tailor it to his own needs. 
Because this approach must be purpose-specific, the following purposes 
can be used to exemplify typical applications: 

1. Identify how a particular policy contributes to the 
completion of some higher level goal through investi- 
gations of: 

(a) Activity coordination : identify the policies and 
activities that serve to coordinate the target 
activity with other activities in service of larger 
KPU goals. 

(b) Resource accessibility : identify the adequacy of 
resources to carry out a particular policy. 

(c) . Policy coordination : . identify how a specific policy 

coordinates a set of policies to specify a given 
activity_jgr to regulate some agent or resource. 

(d) Agent/activity impact assessment : list responsi- 
bilities and concerns carried by a set of agents 
or activities and estimate the impact of the 
target policy on the life of one or more of these 
agents or activities- 

2. Identify the elements of the policy through the in- 
vestigation of: . 

(a) Agent/activity identification ;' list all agents or 
activities having enforcement or compliance- re- 
sponsibility under a given policy. 
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(b) Activity analysis : break the activity specified 
by the policy into its constituent stages and 
elements (policies , agents , and resources) , and 
show how the elements interact as the activity 
is performed. 

(c) Policy/activity design : on the basis of existing 
knowledge, assess each stage of a proposed policy 
or activity in terms of requirements for proper 
functioning. 

(d) Policy identification : list all significant 
policies 

(1) a given agent or activity is responsible 
to enforce. 

(2) a given agent or activity is responsible 
to comply with. 

Identify the impacts of policy through 

(a) Impact identification : map all the activities, 
agents, resources or policies that are (or might 
be) significantly impacted by a given policy. 

(b) Agent/activity impact assessment : analyze the 
effects on a given agent or activity of the 
entire spectrum of policies acting on it. 

(c) Policy dynamics : identify other policies that 
aided or impinged on the developmental process. 

Describe policy simply and clearly through 

(a) Identification of the policy(s), generated at the 
level of the aggregate agent, and specification of 
the stages of the activity. 

(b) Identification of the various stages of the activity 
in order or occurrence, the particular agent re- 
sponsible, and the requisite resources. 

(c) Identification of other policies that affect these 

in their operation. If indicated, describe conflicting 
policy (s) . 
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Project Methodology: Phase 2 

While the tasks to be performed in Phase 2 were no easier than those 
in Phase 1, there are fewer steps to be shown graphically. (Compare the 
new plan shown in Figure 5 with that shown earlier in Figure 2.) 

The goal of our new study plan is to develop an analytic framework 
specifically tailored to describe the role of formal policy as an influence 
on KPU and to draw from it and from the case studies specific recommenda- 
tions to present to NIE. To reach this goal, we will use the case studies 
to develop empirical generalizations that will clarify our image of KPU. 
Conversely our image of KPU at the interim report will provide the basis 
for selecting case topics for further pursuit and assigning specific 
analytic purposes to be tested in each topic. 

Both the case studies and the image of KPU will be used to derive 
the specific expectations to be used in forming the final version of the 
analytic framework. 

Therefore a new system of emphases and organization of tasks emerged » 
We placed more emphasis on the case studies- Because we were now searching 
through the case studies to identify what it was important to say rather 
than selecting what to say using an a priori analytic framework, much more 
-effort and "mucking about" had to go into them. Conversely since most 
of the case study work was preceding the framework, and being used to 
develop (rather than to test) the framework, the cases would not generally 
demonstrate how the framework was applied (except in a post hoc fashion). 

This plan is based on the assumption that the nature of KPU components 
should be taken into account in forming the framework. Specifically some .. 
of the features of these components are that they have parts and relations 
that must be described. They have wholes which have properties that emerge 
because they are wholes. And they exist in a larger education social arid 
physical environment which impinges on them. Moreover different audiences 



FROM PHASE ONE 



CONTINUING LITERATURE 
SEARCH 



CLARIFYING "IMAGE" OF 

KPU AND KPU(G) 
PARTS: Elements and Relations 
that Need to be Included 
WHOLE IN ENVIRONMENT: 
KPU Relationships to the 
Environment that must be 
Described 

PROPERTIES OF WHOLES: 
Effective Levels of Description 
PURPOSES OF ANALYSIS: 
(Monitoring) Information Needs 
of Audience and KPU{G) Actors 

KPU Topics to be Described 



Data for 
Revising 
"Image" 



Criteria for 
Selecting Topics 
and Purpose(s) 
of Analysis 



Application 



EXPECTATIONS 
FOR MONITORING 
PROGRAM 



REVISE REQUIREMENTS 

FOR ANALYTICAL 
FRAMEWORK/APPROACH 



CASE STUDIES 
Description of 
Governance 
Highlight Policy 
Special Problems 
(Governance-related) 



J 



ANALYTICAL FRAMEWORK AND 
ANALYTICAL APPROACH 

• Redesign/Refine 

• Supporting Taxonomy 



Feasibility 



CASE STUDY 
CONCLUSIONS 



Project Goal ^ 



I: 



RECOMMENDATIONS 

• Indicators 

• Design Implications 

• Special Research 
Studies 




FIGURE 



STUDY METHOD IN SECOND PHASE 



43 



will see different components in the KPU field and will use different 
perspectives to describe the same component. 

The analytical framework must be tailored to be useful in face of 
all of these observations. Moreover these expectations would contain 
specific implications for the monitoring program into which the framework 
should fit. 

We stt)pped searching for "the correct paradigm" of KPU and began 
developing a general method of inquiry into KPU governance flexible 
enough to serve even antithetical perspectives and value system.s.' The 
analytic framework was now expected to be a tool for guiding concept forma 
tion rather than for systems mapping of KPU sectors. 

This strategy should permit the incremental development in an NIE 
monitoring program of an ecology and anatomy of KPU so that gradually 
social indicators can be selected and their significance understood. A 
major advantage in using this new strategy was that it was not necessary 
to agree on the basic conception of KPU or how KPU is best portrayed in 
order to proceed. Like the scientific method, it would allow each analyst 
to pursue his own hunches and line of investigation and yet contribute 
overall to an evolving description of the real world. 

The Focus on Formal Policy, a Regulator of KPU 

The shift in the methodology of the study was accompanied by a shift 
in our presumption about the role of policy and formal regulators of KPU. 
Rather than presume that there are distinct regulators of the system we 
asked whether there are distinct regulators of the system. Because our 
formal charge was to describe the role of formal policy in KPU governance, 
we centered our data gathering around policy and used policy archives and 
related records as our basic data source. 
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Our major question shifted from^ How does policy regulate the system? 
to^ How well do the data in policies and policy archives serve as the 
basis for describing the governance of configurations and their infra- 
structures? 

The first cycle through this new plan was conducted to produce three 
outputs: an initial list of the types of specific expectations we had 
for the analytic framework^ a clarification of our expectations for the 
taxonomy^ and a selection of the case topics to be pursued in the remainder 
of the project. 

Expectations for the Analytic Framework 

Three basic expectations for the framework were immediately identi- 
fied. Because we now saw our goal as simply describing KPU, the first 
expectation for the framewprk was "realism." As far as possible the 
reconstructed logic produced by the framework as descriptions of the KPU 
infrastructure had to correspond to the actual logic in use of that 
infrastructure. Second, the descriptions produced by the framework had 
to have pragmatic value or usefulness. It is not enough that the descrip- 
tions be valid; they must have a favorable - cos t-benef it trade-off and make 
a positive contribution to KPU policy planning. Third, the framework must 
have communicability, by which we mean that the models and policy impli- 
cations that are produced by using the framework are communicable.* 
Furthermore, the framework itself must be communicable. How easily can 
the paradigm be taught to other individuals? How can it be incorporated 
into the policy planning procedures of NIE or any other agency? 

We hoped to fill in and expand on these three expectations based on 
the feedback from the case studies and the continuing literature search. 

It is not enough for policy-related results of an analysis to follow 
validly from it. It is important to be able to show that they follow 
validly. 

45 

■ ^ 67 " y 



The Role of the Taxonomy: Making the Analytic Framework 
Teachable 

The new study plan suggested a rather radical shift in our concept 
of the role of the taxonomy. Since we were using the cases formatively 
in the development of the framework we needed a strategy to keep track of 
where relevant policy data were to be found and to help generate appropri 
ate questions to ask of data sources. Conversely, as the framework 
developed we needed a way to record and communicate among ourselves the 
principles that made up the framework. 

The concept of a taxonomy to serve these goals was put forward. 
This taxonomy, organized around policy, would be a heuristic guide to 
relevant policy types and an aid to developing a checklist of information 
suitable to complete an analysis aimed at serving a specific purpose. 

As a heuristic guide to inquiry this taxonomy would contain and be 
organized by the principles of the framework and would thus constitute 
a major tool useful to communicate the framework as a methodology' to 
other investigators. " 

Selection and Orientation of the Case Study Topics 

The final selection of case study topics was conducted with the 
following limitations in mind: 

• Show first and foremost what must be in the framework to 
deal adequately with formal policy influences in the KPU 
infrastructure. 

• Determine how well policies will serve as the principal 
source of information describing the KPU infrastructure. 



In fact, what we finally concluded was that a heuristic taxonomy and 
a method for communicating the analytic framework should bedeveloped 
separately. 
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• Because of limits of resources, provide insights into 
developing a strategy for but not actually trying to work 
out the complex detail of the interaction of policy net- 
works . 

• Span the domain of KPU to expose the range of inferences 
for a monitor program. 

Hence we chose the ten case study topics listed below. 

I. National Institute of Education (NIE) Allocation Policy. 

II. Federal Procurement Policy and Knowledge Production and 
Utilization in Education, 

III. Assessing the Impact of Policies that Control the Availability 
•of Information. 

IV. The Far West Laboratory as a Research and Development Performer. 

V. Minicourses as an Example of Policies Affecting the Dissemina- 
tionAltilization of a Successful R&D Product. 

VI. The School Mathematics Study Group (SMSG) Project as an Example 
of Policies Affecting the DisseminationAf tilization of an R&D 
Product. 

VII. ESEA Title III Teacher Initiated Innovation Program: An 
Example of Policies Interfacing Levels of Government. 

VIIIj The Governance of Knowledge Production and Utilization in 

Intermediate Service Agencies: Boards of Cooperative Educa- 
tional Services in Colorado and New York, 

IX. Policies Affecting the Results of the Federally Sponsored Pilot 
State Dissemination Program in South Carolina: 1970-73. 

X, An Approach to Monitoring the Role of Government Policies in 
the Process for Selection and Evaluation of New Instructional 
Materials. . 

These topics were chosen specifically to cover the range of governance 

agencies from the school site to the state and the federal, levels and the 

types of KPU activity from basic research to linkage and utilization (see 

Table 1, third column). Because we were focusing on tlie formal control 

messages in the system we also investigated the spectrum of policy types 

running from constitutional law to research facility contracts and school 

site memos (see Table 7, fourth column), 
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Because of the many facets of the paradigm to.be tested within a very 
limited budget of resources and time^ the case studies were each assigned 
a specific set of four "purposes of analysis^" one on each of the four 
dimensions listed on pages 46-47, Each case was to show interesting sub- 
stantive insight into an espect of its topic area^ pursue a line of in- 
vestigation that had immediate utility for KPU policy planning^ contribute 
specific insights to formation of the analytic framework; and finally be 
written up in a way to explicate some aspect of the framework as well as 
of substance. Three styles of write-up were chosen^ one to explicate the 
findings of an analysis without revealing much about the framework^ one 
to explicate the findings through the framework^ and one to show specifi- 
cally the steps one might follow in using the analytic framework. 

The Imase of KPU Takes Shape 

During the remainder of the project the case topics were gleaned for 
specifics to flesh out our image of KPU and thereby to contribute specific 
expectations for the analytic framework. The image that resulted is sum- 
marized in the following pages. It is in response to this image that the 
particular principles that constitute the analytic framework were chosen. 

The "KPU" that can be empirically observed is a set of behaviors or 
activities. They are identifiable as KPU activities by the purposes for 
which they are conducted. The properties of these activities can be 
grouped into three classes; some pertain to the parts (the elements and 
relationships) that come together to accomplish the activity, some pertain 
to the activity as a whole and are what cause us to see the activity as 
a whole (e.g., selection of new textbooks), and some pertain to the 
relationship between that activity and its larger environment. The 
specific outcome of an activity is affected by formal policy regulators 
acting on it from any of these three perspectives. Some- policies regulate 
components (e.g., what must be included in the content of textbooks), some 
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regulate the whole activity (e.g., textbook selection must be conducted 
periodically), and some regulate the environment of the process thereby 
affecting textbook selection (e.g., minorities must be treated with 
proper respect) . 

This distinction of the three ways of knowing a particular KPU 
behavior provides a convenient organizer for presenting not only the 
specific features of our concept of KPU but also our expectations for th 
analytic framework. 

Observable Elements of KPU 

What can be directly observed and measured as the building blocks 
of KPU is not the infrastructure nor the processes as such. It is the 
people acting as KPU agents, the " policies'' they transmit as messages to 
regulate or coordinate their efforts, the resources they have at hand or 
declare they need and the specific activities they undertake pursuant to 
some KPU relevant goal. 

Observations About Agents in KPU 

1. Most agents who perform KPU tasks are not and do not 
consider themselves first and foremost KPU agents. 
KPU is a sideline or small component of their overall 
role (e.g., all teachers are to some extent KPU agents). 

2. On the other hand there are certain classes of agents, 
particularly at the SEA and federal levels, who depend 
entirely on federal dollars to continue their KPU tasks. 

3. Agents report relatively little dysfunctional con- 
straint by policy overload (except those agents who are 
trying to make changes from a position outside the for- 
mal structure). Some agents, in fact, reported both 
policy overload and policy conflict to be a useful tool 
by which they could expand their own domain of discre- 
tion without increasing risk. 
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4. Agents report generally adequate paths to resolve 
policy conflicts but seek to resolve the conflict 
only when it works to their advantage. Often 
policy conflict can be used by agents as a shield. 

Observations About Policies in KFU 

1. Most of the policies which affect KPU are not first 
and foremost KPU policies. Usually they were formu- 
lated to pertain to either main line educational 
practice (e.g,, the Buckley Amendment) or larger 
issues (e.g,, desegregation). 

" 2. Policy acts in at least five equally as important 
capacities in the system as that of regulator. 
Policy 

(a) defines agent roles. 

(b) establishes domains of discretion. 

(c) sets the agenda of concerns. 

(d) serves as a symbol of compromise and as a 
rallying point for constituencies. 

(e) serves as an archive and historical record 
of agreements and bargains. 

3, Myriad^ perhaps most of the policies or norms regulat- 
ing KPU are informal^ cultural^ and unwritten. Of tho 
that are formal^ only a few percent ever surface as a 
visible source of constraint. KPU is not mainly regu- 
lated by that small visible fraction. (This small per- 
centage is still very numerous in absolute terms.) 

4. Real conflicting policies and goals (or at least inter- 
pretations thereof) abound^ especially in the informal 
domain. 

Observations About KPU Purposive Activities 

1. Most of the purposive activities marked with some KPU 
aspect are not primarily KPU activities (e.g., improve 
reading performance).' 

2. KPU and education involve some of the most strongly 
held values and goals as the basis for activity. 
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3. There is important and irreconcilable conflict 

among these goals and values that can be managed 
in a peaceful way only through political means. 

Observations About KPU Resources 

1. Most of the KPU resources formally labeled as such 
flow through NIE and other federal agencies. 

2. However the majority of resources used for KPU pur- 
poses are not allocated as such. (e.g., local district 
allocation for implementation of specific new pro- 
grams). 

3. . The most easily identified resources are money and 

personnel. 

The Molecules of KPU 

One of the major lessons of the case studies is that, since the prop 
erties of the components used in KPU make them even more commonly useful 
for other social purposes, it is virtually impossible to tell a priori 
what the KPU relevant policies, agents, and the like are. It is only 
when these components come together in particular ways that they are KPU 
or KPU relevant. Hence we decided that for purposes of analysis and moni 
toring there is a smallest int'felligible unit* of KPU that is recognizable 
as KPU. This unit is composed of an arrangement of some of each of the 
four basic components. From the standpoint of analysis these units are 
recognizable because their manifest behavior pursues some KPU goal. By 
identifying the unit(s) pursuing specific KPU goals it is possible to 
identify and interpret the meaning of quantitative descriptions of their 
components and interpret the significance of the prevalence of those com- 
ponents in the general environment.* 



An understanding of the structure of KPU units will make possible the 
selection and interpretation of useful social indicators. 
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Because the KPU goals and values held by different groups vary 
greatly and often are in real corflict, however, the identification of a 
unit from one perspective will often not tnatch that from another perspec- 
tive. Likewise, depending on the shift in perspective, descriptions of 
units will overlap, be contradictory, and sometimes leave gaps when 
brought together to create a whole picture. 

Some order can be brought to this maze of conceptual units for seeing 
KPU by recognizing and utilizing the multitiered formal structure for 
organizing KPU management as one principle of the analytic framework. 

The Ecology of KPU 

In addition to characteristics of its parts and basic units the KPU 
infrastructure is a product of the environment, particularly its immediate 
environment. Again the case studies revealed a series of specific obser- 
vations about this relationship. 

1. KPU is subject to appeal by special interest groups and 
shifting coalitions within the political and social 
systems that are outside KPU. 

2. By law KPU cannot be directly manipulated from the federal 
level except through incentives. 

3. The fiscal incentives made available to federal KPU pro- 
moters are extremely limited. 

The Domain of Inquiry 

Thus, we came to a basic conclusion about the KPU infrastructure 
as a domain for inquiry, monitoring, and management. We should not assume 
that it is one system except by definitional tautology. It is a maze of 
components only partially ordered and often only connected by a shared 
environment. We thus agree with the following statement made in the 
recent NIE funding policy report. 
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Systeni Qualities 

... we want to comment on the degree of "system" that seems to 
exist. We have been impressed, even though we have far too 
littl'e data, to find so little interrelation of parts. There 
is an aggregate of individuals, institutions, and other re- 
sources that combine and relate in a variety of ways to a 
variety of ends .... But it also reinforces our conviction 
• that considerably greater effort must be put into mapping the 
pieces of the universe, understanding the diverse structures 
and their internal strengths, so that deliberate policy toward 
each part can be attempted, and so that potential connections 
and interrelations can be encouraged. 

The analytic framework (National Institute of Education, 1975, p. 30) must 
be a method for constructing such descriptions. 

Some of the more salient features of KPU that must be taken 
into consideration to develop an analytic framework to describe KPU 
sectors are listed below: 

1. By starting with a focus on LEA and school site 
staff we were driven to trace back the lines of con- 
straint and pattern of decision making of which 
these agents are only the final gatekeepers. The 
pattern behind these visible agents explains KPU 
outcomes, just as the bulk of an iceberg beneath 

the water explains why what is visible floats. 

2. KPU is therefore hierarchically but not centrally 
controlled. 

3. On the other hand a strategy has not yet been 
developed to manipulate the KPU hierarchy to serve 
specific goals. Therefore much of KPU is con- 
ducted laissez faire. 

4. The image of educational KPU is highly unstable 
among both professional staff and laymen. Each 
shift in image brings a change in strategy. 

5. Governance is not a simple process consisting of 
those who govern and those w.ho are governed. We 
found that we had to consider governance, or regu- 
lation in the .political sense, a complex process 
of negotiation between many different groups with 
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different positions in an authority structure, 
with' different needs and perceptions of how those 
needs could be met, and with different resources 
at hand. Governance is diffused throughout the 
system with all agents having at least some dis- 
cretion. 

6. There are real sources of indeterminacy in the KPU 
infrastructure springing from the following sources: 

(a) The pecularities of human agents acting within 
their role-defined domains of discretion. 

(b) The limits of skill, intelligence, and fore- 
thought of human agents. 

(c) The diversity of goals and values. 

(d) The possibility of direct violation or igno- 
rance of policy directives. 

(e) The lack of tools to act on or enforce 
policies . 

(f) The artificial nature and therefore potential 
contradition and ambiguity of policies. 

Thus, overall, the cases show that KPU does not easily submit 
to simple preconceptions of how it works. It is necessary to look at 
KPU practice in some detail to construct a realistic model of how KPU 
behaves. 

Expectations to Use as Design Criteria for the 
Analytic Framework 

From the image of KPU developed through the case studies we derived 
the following system of expectations to use as design for the analytic 
framework. First we formulated expectations for the framework as a 
distinct methodology, second for the role of the framework if it is to 
serve as part of a monitoring program, and third for the specific prin- 
ciples that should constitute the framework. 
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The Analytic Framework as a Distinct Methodology 

The framework itself would be best characterized as a metaguide 
to inquiry into the KPU governance structure just as the scientific method 
is a metaguide to inquiry into nature. Therefore, the framework should 
have as its conceptual base the way we can '^know useful things" about the 
KPU infrastructure (not some specific template of that infrastructure). 
It must be able to cope with the indeterminacy and disorder actually found 
in KPU and do so in a systematic way. Unlike the physical scientist who 
assumes that nature is well ordered, the student of KPU cannot make this 
assumption about KPU. 

Moreover when a scientist constructs a framework for inquiry 
into the physical world and oversimplifies the problem, nature is not in 
fact changed to conform to that image- KPU, however, is an artificial 
system that reacts to congressional mandates", shifts in public expecta- 
tion, and major infusions of money. An overly simplified strategy for 
mapping KPU could lead to actions that narrow rather than broaden the 
range of change KPU produces. The practice system has been dominated by 
the search for the "one best system" long enough. To promote change, the 
KPU monitoring program must be able to identify and support even more 
variety than currently exists and to embrace even more variety as the 
education system becomes more pluralistic."' 

While the analytic framework is to be designed to cope With the 
formally specified infrastructure, it should also be designed to leave 
room for eventual inclusion of the informal regulatory structure and for 

This observation incidentally points to a basic flaw in conceptualizing 
the control-of -schools issue as a centralized-decentralized governance 
issue. Without adequate differentiation at the local level there is no 
need for central support; without central support there cannot be dif- 
ferentiation at the local level. 
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use in conjunction with the other facets of a programatic monitoring 
program. 

The framework should provide a method for describing specific 
units of the KPU infrastructure relative to various purposes and do so 
in an efficient and cumulative way. This methodology should serve as a 
rigorous, consistent, and communicable tool for revealing the connected- 
ness and pattern of KPU as it really is. 

The Conceptual Role of the Analytic Framework 
in the Monitoring Program 

Incidental to the formation of this system of expectations for 
the framework, we began to see their implications for a monitoring pro- 
gram. In turn these implications affected our expectations for the 
framework. We began to believe that the monitoring program must be 
designed to reflect the understanding of the substance and process of KPU 
and not simply of some highly abstract and general concept of monitoring. 
The monitoring program ought to be designed by looking at the details. of 
the KPU process. Moreover, the program ought to have a built-in capacity 
to permit structural modification as the image and understanding of KPU 
develops. 

In Figure 6 we present our image of a monitoring program in 
contrast to that seen in Figure 1. 

Thus, in summary, the role of the analytic framework in a 
monitoring program is to: 

1. Provide a strategy to ask ''good" questions about how 
the infrastructure leads to particular KPU outcomes. . 

2. Provide rules for organizing answers into policy 
useful patterns that enhance KPU efficiency. 
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Expectations for the Specific Principles 
Composing; the Analytic Framework 

Earlier we mentioned that the framework ought to be able to pro- 
duce realistic and useful descriptions of the KPU infrastructure. We are 
/ 

now in a position to use our image of the empirical reality of KPU and 
our expectations for our monitoring programs to explain what these mean 
in greater detail* Table 8 contains a simple outline of what we mean by 
realism and utility . We discuss these concepts below. . 



Table 8 

CHARACTERISTICS OF AN EMPIRICALLY VALID 
ANALYTIC FRAMEWORK FOR INQUIRY 



Realism 




Utility 




Comprehension 




Pragmatic usefulness 




• Complete 




• For a variety of 


purposes 


• Complex 




• Different levels 


of effort 


• Integrative 












Communicability 




Balance 




• Of results 










• Among four primitives 


• Of paradigm 




• Among three types of knowing 






• Social versus 


personal goals 






• Formal versus 


informal norms 







Re alism --The most devastating critiques of the prevailing models 
of educational R&D used to map the KPU infrastructure are that the models 
sacrifice important real characteristics of KPU in order to have logical 
simplicity, or mechanistic linearity, or some other ty.pe of conceptual 
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neatness. This sacrifice is a mistake not because it is an attempt to 
find ways to conceptually simplify a complex field; rhe mistake is in 
using, a priori, a simple conceptualization as a sort of procrustean bed 
for understanding KPU. In fact, rather than applying a model to KPU, a 
paradigm is needed to abstract models from the empirical reality of KPU. 
To be realistic or valid that paradigm should be molded to exhibit two 
particular properties: comprehension and balance. 

By comprehensive we mean that a realistic framework must have 
the ability to take advantage of or incorporate all elements which may be 
important to explaining different KPU events. The shift from educational 
R&D was one shift in this direction, showing that dissemination and adop- 
tion activities needed attention as well as research and development. 
Currently there is pressure to incorporate the concepts of linkage and 
utilization into the basic linear model. 

Our suspicion is that even this is just the beginning of expan- 
sion of the basic model. KPU is a small sector of the $100 billion a year 
American education industry. KPU as an industry is only beginning to take 
shape. A strategy to model that industry must have room to incorporate 
much more complexity and variety as the industry develops, otherwise the 
models will continually fall under their own weight." 

Therefore for the framework to lead to comprehensive models it 
must answer questions in three areas (see Table 8). Does the framework 
produce descriptions of KPU that are complete, appropriately complex, and 
adequately integrative? 



The capacity to cope with variety is especially necessary when we observe 
that the goal of KPU is eventually to anticipate, not just respond to 
necessary change. KPU must make possible more varied responses in the 
education system and not limit responses to those easily modeled. 
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By balance we mean that a realistic analytic framework must 
balance its treatment of the effects of the different factors in KPU. In 
this case four dimensions are of particular concern (see Table 8): the 
role of formal versus informal norms and policies, the role of social 
versus personal norms, the role of three ways of '^knowing" a KPU (parts, 
whole, and ecology) and the role of policies versus that of agents, re- 
sources, and activities. These elements comprise the primitive components 
of KPU. 

As we interpreted the purpose of this project it seemed that 
the central task was to focus the bulk of our efforts on determining the 
role of formal policy in regulating KPU behavior. In this sense our 
efforts were unbalanced. However, as the reader can see from the precedin 
paragraph, this is only one type of norm regulating system behavior; 
moreover, norms of all types are only one of the characteristics of'con- 
figurations that effect configuration behavior, ' Thus we left room for 
development of these other dimensions as we designed the basic structure 
of our framework. 

Utility ->"It is not enough that the framework be capable of 
supporting realistic and logically valid descriptions of the KPU infra- 
structure. Both the descriptions and the framework that produces them 
must have pragmatic utility or value. This utility must be of two basic 
sorts: practical usefulness and communicability (see Table 8). 

By practical usefulness we mean that the framework must facili- 
tate understanding of KPU configurations in cost-effective ways that are 
relevant to policy planning and can lead to better decision making. For 
the framework to be facile in this way it must produce conceptual con- 
figurations that reveal valid policy-relevant implications, do so effec- 
tively, and meet a wide variety of analytic problems faced in policy 
planning and assessment. 
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By communicability we mean two distinct things. First, we mean 
that the configurations and policy implications that are produced by using 
the framework are communicable. . That is, it must be possible to present 
the results of an analysis to a policy maker in a coherent understandable 
way. It is not enough for the policy-related results of analysis to follow 
validly. It is important to be able to show that they follow validly. 

Second, the analytic framework itself must be communicable. How 
easily can the paradigm be taught to other individuals? How can the 
paradigm be incorporated into policy planning procedures of NIE or any 
other agency? 
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Ill AN ANALYTIC FRAMEWORK' FOR GUIDING DESCRIPTIONS 
OF THE KPU INFRASTRUCTURE 



Overview of Section III 

In this section we present the analytic framework as a response to the 
expectations it must meet. These expectations are dictated by the KPU 
reality and the practical limits of a monitoring program. The analytic 
framework is a method for guiding inquiry into KPU in education tailored 
to capture the salient features of the KPU infrastructure. These features 
are clustered into three sets; those pertaining to the elements and basic 
relationships that compose the infrastructure, those pertaining to the 
**wholes" in the infrastructure, and those pertaining to the relationships 
between "wholes" and their environment. In each of the three cases the 
most notable sources of indeterminacy are presented. The final portion of 
this section presents two taxonomies that make the framework easier to use 
and an approach that makes the framework teachable. 

The Analytic Framework; a Gestalt 

Based on the preceding argument we decided that the framework should 
be a strategy for concept formation where the subject of the concepts, is 
the KPU infrastructure. In this respect we will call the basic unit of 
understanding con f i gur a t ion . * Configurations are conceptualizations of 
the KPU units. They are the descriptions of particular KPU phenomena 
constructed to bear the highest possible correlation to the logic in use 
as seen from a particular perspective to serve a particular purpose. 

The features and distinctions used to construct configurations can 
be conveniently grouped into three types; the first pertain to the parts 
and relationships that make up the "whole," the second pertain to the whole 
that make it more than the sum. of its parts, the third pertain to the whole 
as part of a larger environment. 
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We will say simply that the framework must help promote inves tigating , 
knowing, and understanding a KPU configuration by 

• Recognizing its parts or components and the ways in which 
they are related, 

• Recognizing the emergent properties that make the "whole 
more than the sum of the parts." 

• Recognizing the relationships between the unit being con- 
figured and its environment. 

There is a great deal of difficulty in drawing objective boundaries 
around social enterprises — saying what is within a configuration and what 
is in its environment. But the difficulty dissolves when the choice is 
left entirely up to the analyst as guided by his analytic purpose. The 
boundary distinction then rests on the criteria he or she uses to value 
components for inclusion. The boundary thus drawn is neither true nor 
false, merely more or less useful. 

Boundaries of configurations (like those of any concept) are there- 
fore merely artifices. We draw them to limit a description and to arrange 
elements economically and usefully. They are highly permeable lines of 
demarcation. There is no hard and fast rule for defining boundaries 
other than the general considerations of usefulness. 

The purpose of the analyst has become the central standard by which 
investigation and adequacy of description are to be measured. The purpose 
of analysis* takes its central importance because it is the "value" that 
enables the analyst to distinguish what is the subject of his inquiry 
and to sort the focus of attention from the surrounding field. 

We have chosen to call the subject matter of a configuration the 
focus of analysis . This focus of analysis is distinguished from the 
surrounding field of analysis by the purpose of the analyst. 



Note that the purpose of analysis we are using here is the purpose of the 
analyst, not of the KPU agents being studied. Organizing analysis around 
hypothetical purposes of KPU agents such as RDDA is like seeking diagnosis 
for one's own ills by reporting someone else's symptoms. 
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Sources of Indeterminacy 

Before we explain the principles of the analytic framework that are 
suitable to guide the three sorts of inquiry into configurations, we wish 
to point out that there are two fundamental sources of indeterminacy that 
will plague each and every attempt at description. 

The first source of indeterminacy has to do with the profound limits 
of mankind^ s basic tools for description: logic and language. When dealing 
with complex man-made systems, from mathematics to language to systems of 
social choice, attempts at complete, consistent, and realistically complex 
description always turn up irreconcilable paradoxes in the descriptions 
themselves- — declarations or descriptions constructed with perfect fidelity 
to the rules of reason but mutually contradictory. This first type of 
indeterminacy plagues every tool of inquiry man has designed. The limits 
of the tool should therefore be understood so as not to be confused with 
real limits in the reality studied. 

o 

The second source of indeterminacy is much more common and of more 
immediate concern. Some of the features of KPU components are unpredic- 
table either because we haven't the skill or resources to make them predic- 
table or because the phenomena under study is a source of idiosyncratic 
behavior. (For example, the specific next act of a human being is. not 
reliably predictable. Faced with a policy directive, a person has many 
options within the scope of the policy as well as the option of doing the 
forbidden.) 

In each of the following sections we briefly discuss the kinds of 
indeterminacy that must be coped with in that type of description. 

Parts of Configurations 

The real world observables in any KPU focus of analysis are agents, 

resources, policies, and purposive activities. From these we infer patterns 

and flows of influence, and deduce cause and effect. 

The primitive components of KPU are agents , policies , purposive 
activities , and resources . With these four components we found it possible 
to complete a realistic description of any KPU configuration. These four 
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components are not merely convenient constructs like those of '*RDDA" but 
have unambiguous referents in the real world. It may be argued whether 
a particular agent ^person or institution) is a KPU research agent, but 
the existence and identification of an agent is subject to physical de- 
termination. Likewise, while individual policies may or may not apply to 
specific KPU phenomena, the existence of formal policy and its formal 
interpretation is usually straightforward. Where it -is not, systems of 
due process exist to remove any dysfunctional ambiguity. Resources (mate- 
rials, physical plants, money, information, and the like) and purposive 
activities (selecting textbooks, running a regional lab, and the like) 
share the same sort of physical unambiguity- " 

The ambiguity comes in determining their relevance to KPU and, as 
we have argued, relevance should be determined relative to the analyst's 
purpos e. 

It was proposed by several reviewers that a fifth component, namely 
media, needed attention. We considered this possibility but, while 
endorsing the idea that media must be accounted for, rejected it as a 
distinct component. 

There are explicit communication patterns within the system 
that we call media; these include the carriers of policy and 
informal criteria. Agents, policies, resources, activities 
and their patterns are the media for information regulating 
KPU. 

Agents 

Agent is a generic term used t6 identify both individuals and 
organizations at any useful level of aggregation. Agents are identified 
by reference to their location and their formal KPU roles. 



Because we found time to be more usefully conceptualized in KPU descrip- 
tion in other ways we do not treat it as a resource. It is rather a type 
of constraint that can be taken advantage of (e.g., shifting deadlines, 
sequences) but cannot be manipulated directly the way resources can. 
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Policies 

A policy as a part of a configuration is a decision rule. In 
this case we focused on formal policies. It Is evident that policy Is 
only one of the "things" to know In order to "know" a configuration. How- 
ever^ configurations may theoretically by knowable by looking only at formal 
policy if there is sufficient information redundancy in KPU policies. 
That is, if policies are not only decision rules but also symbols of 
agreement, records of structure, and the like, then looking at formal 
policies may be enough to map some configurations. 

Policies are symbols of action In three ways. Policies are 
set, enforced, and Implemented. 

We make a distinction between three roles of policy: 
policy setting , policy enforcing , and policy Impll- 
mentlng . Policy setting Is carried out by agents 
who make policy; enforcing Is done by those agents 
either so designated In a policy to ensure compliance 
or through various Informal sanctioning powers; and 
Implementing Is the doing of goal-oriented activities 
■within degrees of freedom set up by some policy (s). 

Recursion enters here because setting and enforcing are often 
types of Implementing. Thus the policy process must be relatlvls tlcally 
described. How the distinction is applied is determined by the boundaries 
of the foci and field of analysis at the given stage of inquiry. 

We found ideas of setting and implementing the most useful to 
identify relevant policies and classify their effects. "Enforcing," how- 
ever, is a major source of indeterminacy in describing policy. Why a 
policy is effective as a decision rule is largely determined by informal 
incentives and constraints. 

There are Informal Influences that may be more significant 
than formal policies In regulating KPU. Formal policies 
often permit a wide latitude of discretion for implementers 
that Is narrowed by these Influences. This concept is in- 
cluded to allow further expansion of the framework to a 



Policies also pertain directly to whole KPU conf iguration6 (e.g., by-laws 
of a research lab) and to overall relations between configurations and 
their environment (e.g., copyright). 
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more generally useful and realistic understanding of be- 
havior in a policy-regulated social system. 



Purposive Activities 

The activities we are concerned with are those which have a KPU 
purpose as at least one of their motives. Because luany activities in this 
field are intended to promote change and break educational stereotypes 
from the past, it is very difficult to build a descriptive typology of 
activities that does not reify the categories and thereby limit description 
of the variety that exists in the real world. 

Activities exhibit the property of process in time. Therefore 
we found it desirable to distinguish how time should be used to describe 
activities. In the first place, activities evolve. The basic task changes 
with history. Second, and more important for short-term projections, the 
activities that contribute to a configuration are typically ordered in 
some sequence and bear a time sequence among them that is necessary to 
their smooth contribution to the whole. 

Several concepts of time are necessary to describe the 
parts of configurations. First, activity sets evolve 
through time; the functions of organizations change 
as goals and objectives are redefined. Second activi- 
ties are often rearranged in sequences. 

Treatment of the time sequence of activities suggested the next 
major distinction to be made in the analytic framework; that between an 

* 

activity and the component activities that make it up. 

It is sometimes useful to make a distinction between 
activities and levels or stages of activities . Policies 
that govern KPU programs frequently make this dis tinction-- 
various subactivities are prescribed to be performed par- 
allel or in sequence. Within organizations, various bureaus 
carry out activities that contribute to the main purposive 
activity(s) of that organization This distinction creates 
the necessity for concepts of hierarchy for agents, activi- 
ties, and policies. 

It is this notion, that activities occur as stages of larger ac- 
tivities and in turn are composed of stages of smaller activities, that we 
found most useful in developing ordered aggregation and disaggregation of 
KPU wholes. 

70 

92 



Resources 

Resources are constraints; material, energy, and the like that 
arise from the physical world or are information about that world. They 
are the stuff of which KPU units in the real world are built. Because 
they act on KPU in a different way from policy we did not spend our ef- 
forts on elaboration of their role. We did, however, attend to how policy 
affects the allocation of resources. 

In summary, a disaggregated configuration is composed of "smaller 
or component configurations and/or a set of agents (with roles defined by 
formal or informal norms), organized to conduct a purposeful activity and 
"related" in four ways: 

1. By shared formal and informal policies (including 
those which define agent roles). 

2. By exchanges of resources , 

3. By exchanges of information, and 

4. By a shared environment. 

Fundamental Sources of Indeterminacy in the Parts of Configurations 

The principal sources of indeterminacy in describing the parts of 
configurations arise from two sources: agents and policies. Agents are 
sources of indeterminacy in that they are roles played by people. People 
have their limits and flaws as a source of indeterminacy, but more 
importantly they always act within some range of discretion even when they 
are "going by the book." Moreover there is no guarantee that they will 
go by the book and often they choose not to. 

Policies are a source of indeterminacy because they are a set of 

man-made rules which are not of necessity complete or consistent. Policies 

often are ambiguous, misinterpreted, and deliberately vague. They are 

based often on real value conflicts and are sometimes legitimately in- 
terpreted in one way today and in the opposite way tomorrow. 
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Configurations as Wholes 

A configuration is a whole that is more than the sum of its parts. 
It is identified as a whole principally for one or more of three reasons 
of interest to us: 

• It is treated as a whole by some formal policy (it is 
formally recognized as one thing). 

• It exhibits emergent properties that are not merely 
aggregates of the parts (such as one coherent purpose 
or integrated set of purposes). 

• From the perspective and purpose of the analyst it is 
conveniently treated as one unit to promote understanding 
and communication. 

We generally found it possible to locate an important policy that, 
implicitly at least, marked distinctions to use in identifying wholes. 
In general such policies specify a particular purposive activity that is 
the characteristic emergent property that delineates a whole (e.g., 
curriculum development). 

The treatment of wholes required adding several conceptual tools 
to the analytic framework. First is the idea of hierarchic conceptual 
ordering. In discussing parts we labeled one type of part "purposive 
activity." When organized in a configuration a set of activities arranged 
in the appropriate sequence can be conveniently conceptualized as one 
activity (just as the actions of 22 men on a football field in a down 
constitutes one play). This opens the door to recognizing that the KPU 
infrastructure is aggregated hierarchically. What is seen as a series 
of parts by one actor is considered as one thing by another. To cope 
with this problem of hierarchic aggregation we used "stages of activity." 
Each configuration can be said to be organized around one activity (e.g., 
textbook selection). That activity is composed of some sequence of smaller 
activities at a lower hierarchic level and each of those activities can 
be used to identify a configuration at that level. Likewise each activity 
is part of a large:r activity at a higher hierarchic level, and that higher 
level activity can be used to distinguish a configuration. 
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Second, configurations as wholes not only share the evolution property 
of time as do their parts but also require adding a new dimension of time 
to the framework. Whole configurations of ten exhibit the property of 
cycles in time. Some have developmental life cycles, while others per- 
petually reiterate the same pattern in time. Hence time applies to whole 
configurations not only as evolution but also as cyclic. 

Third, policy is not simply a component of configurations; it can 
take whole configurations as its object. 

In terms of what is known about configurations as "wholes," 
the principal characteristic of configurations is their 
emergent human purpose. This purpose need not be singular 
and usually is not. (Configurations have such purposes as 
survival and security as well as the pursuit of particular 
acts such as research , development , dissemination , or 
adaption.) Finally, whole configurations are the objects of 
some policies. 

Fundamental Sources of Indeterminacy in Configurations as Wholes 

There are two fundamental sources of indeterminacy in describing 
configurations as wholes. The first is a subtle and debated point in 
the philosophy of scientific inquiry, namely, that the emergent properties 
of a whole cannot be deduced from. properties of the parts. This means 
that we can expect wholes to exhibit properties only visible by examing 
the whole. Often these properties are exhibited only when the whole 
interacts with its environment. The problem of indeterminacy arises be- 
cause the higher the whole is in our conceptual system (as wie focus on 
larger and larger wholes) the harder and riskier it is to manipulate 
it experimentally and the fewer are the number of instances to observe and 
compare without manipulation (e.g., while there are millions of classrooms 
and tens of thousands of districts there are only 50 SEAs and one Congress). 

The second source of indeterminacy about wholes is their purpose, or 
set of purposes. The purpose is deliberately vague. The system is regu- 
lated by successively dividing up the domain of discretion and allocating 
parts^ of the domain to lower aggregation levels . Put another way, higher 
order purposes and goals are abstract symbols of a range of acceptable 
specific outcomes. Therefore purposes of whole configurations are somewhat 
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ambiguous by deliberate intent. Moreover, at every hierarchic level there 
is real conflict about both goals and means. One reason for hierarchic 
structure is to resolve only those issues that must be resolved at higher 
levels. This must be done in a sufficiently ambiguous way so that the 
concerned constituency will not split into warring factions and yet in 
a sufficiently specific way to enable lower level configurations to act 
in coordinated fashion. 

Configurations and Their Ecology 

The final set of conceptual tools we added to the analytic framework 
were generated in response to the fact that KPU configurations are always 
found within a larger environment and often that environment accounts 
directly for the behavior of the configuration. 

The environment affects configurations both at the level of its 
parts and at the level of the whole. The environment is a major source 
of people, resources, and policies that are or affect parts of configurations. 
On the other hand, the environment creates a niche for the whole configura- 
tion and is a source of policies aimed at the whole. Because the environ- 
ment evolves through time, the constraints and expectations it makes on 
configurations change (e.g., Sputnik in the 1950s, the "Great Society" 
in the 1960s). Finally the bulk of the informal norms and policies that 
affect configurations and the majority of the physical parameters that" 
constrain the configuration are considered to be part of the environment,. • 

But the most significant new concept that must be added when we in-- 
elude environment in our thinking is that the environment may be the major 
media by which two configurations communicate. This does not make the 
environment part of a larger KPU configuration any more than communication 
by phone makes the phone system part of KPU. The environment provides 
paths "outside" KPU between KPU configurations. In fact it might be 
plausibly argued that KPU is a set of pockets of activity connected only 
by a non-KPU environment. 



Eco logy is used rather than environment to stress the connectedness 
rather than the distinction between a/ configuration and its surroundings. 
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This should not necessarily be considered a flaw. Such isolation 
would allow radically contradictory ideas to exist within the same society 
in peaceful coexistence rather than continual conflict or forced resolu- . 
tion through compromise. 

From the standpoint of how configurations are to be known 
relative to their environment they have several features. 
By using the "purpose" of the configuration, it can be 
seen as a focus within a field of various forces. Where 
the configuration is tightly knit into a larger configura- 
tion, one dimension of its environment can be shown by 
picturing the configuration as one stage of this larger 
configuration. (Conversely, when a configuration is 
composed of several "smaller" configurations, it may be 
useful to show these as stages of the larger configuration.) 
Tlie ways in which configurations "relate" to one another 
or with or through the environment is by an exchange of 
those same elemental components that make up a configuration 
(namely exchanges of agents, resources, information, etc.) 



Fundamental Sources of Indeterminacy 
in Configurations in an Ecology 

Two sources of indeterminacy in describing a configuration arise from 
its environment. The first is that from the perspective of KPU the major 
changes in the larger environment must be taken as unpredictable shifts 
in parameters. Monitoring KPU itself will not anticipate these although 
it might detect early impacts of such- changes on KPU. However, if KPU 
is to anticipate changes in the larger society in order to have adequate 
lead time for reaction^ the monitor will have to reach outside the domain 
of KPU. 

The second source of indeterminacy is the effect of the environment 
on messages between KPU sectors when the environment is the media. 



Summary of the Paradigm 

The following outline shows the key ideas and principles which make 
up the analytic framework. Because the framework is a strategy for 
guiding inquiry into and for concept formation about the infrastructure 
of KPU, the purpose of analysis established by the analyst provides the 
standard or cutting edges for making distinctions and' drawing connections 



and inferences. The first distinction the analyst makes is between the 
focus of his interest and its surroundings, second distinction is to 

what extent he will examine the parts, whole, or environment of the con- 
figuration. The remainder of the framework is a set of principles to 
organize these three kinds of inquiry. 



SUMMARY OUTLINE OF THE ANALYTIC FRAMEWORK 

A. The analyst selects and refines a purpose of analysis which serves 
as the standard for making necessary distinctions and assessing the 
progress of the analysis. 

B. The analyst distinguishes (based on his purpose of analysis) between 
the focus of his investigation and field of phenomena in which it 

is found. 

C. A conceptual configuration of the focus with three types of features 
is elaborated as needed to satisfy the purpose. These features 
include: 

1. For parts of the configuration 

a) Agents, resources, purposive activities, and Policies. 
(Policies are characterized by being set, enforced, and 
implemented.) 

b) The four basic components and the patterns they form are 
the media or communication system of KPU. 

c) Informal and personal norms outweigh formal policies in 
determining the specific choices of KPU agents. 

d) Time marks the parts of configurations in two ways. They 
evolve and they may be ordered in some sequence. ^ 

e) Specific KPU activities can be usefully treated as stages oi 
larger KPU activities. 

2. For configurations as wholes 

a) A configuration is a whole for monitoring purposes if: 

(1) A formal policy treats it as whole 

(2) It has distinct emergent properties 

(3) For the given purpose of analysis it is sensible to 
treat it as a whole. 

b) The activities used to distinguish full configurations are 
best treated as hierarchically nested, 

c) Whole configurations often have characteristic cycles in time. 

d) Policies are not merely components of configurations but may 
take full configurations as their target. 

3. For configurations within an ecology 

a) . The environment influences a configuration both through 

the parts and the whole. , 

b) The non-KPU environment provides media through which KPU 
information flows. 



The advantage of this analytic framework is that it allows a re- 
searcher to study units and form configurations without conceptually 
destroying the unit (as summary statistics do) in the prpcess and forcing 
information into procrustean molds or categories. 

The analytic framework is a first attempt to form a methodology to 
produce systematically a reconstructed logic of KPU that is congruent 
with the logic in use. It does not try to force/reify systems logic as 
the logic in use. 

Final Taxonomy: a Guide to Policy Inquiry 

We developed two taxonomies, one of available types of policy archives 
and what they contain, the other of configuration- relevant terms and con- 
cepts to help organize the questions analysts should take to the policy 
archives and other information sources. 

Taxonomy of Policies and Their Archives 

This taxonomy and discussion was organized to show the range 
of laws and public policy regulations which are perceived to have relevant 
impact on the way in which KPU is carried out and how innovations are 
implemented in education systems (see Table 9). Secondly it is intended 
to provide some information about how policies are made and where they are 
stored in legal and other archival systems. 

Taxonomy to Guide Policy Search 

While most policy is recorded In accessible archives of one sort 
or another, there is a serious problem in locating the right archive and 
the right index term to retrieve the item. Moreover, organizing the over- 
all "search to make it effective and complete requires some planning. The 
second taxonomy is aimed at helping to meet these needs. 

The taxonomy is basically a checklist structured so that an 
analyst can use the following four general pieces of information in his 
configuration to identify specific policies that might be important to 
the configuration: 
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Table 9 



A LEGAL TYPOLOGY OF POLICY 



Type of Law 



Constitutional 



Statutory 



Administrative 



Administrative 



Administrative 



Contract 



Case 



Case 



Statutory 



- T y p e - o ?ol± c y 



Constitution 



Enactment 



Regulation 



Guideline 



Discretionary Act 



Contract 



Common Law 



Order 



Plebiscite 



De"f inirtion 



A system of fundamental laws or principles for the 
government of a state^ society, corporation or 
other aggregation of individuals. A constitution 
is always written and incorporates as its own all 
final individual decisions made by courts of law 
based on its construction. This concept embodies 
not only those documents widely recognized and 
called constitutions, but also the statement of 
purpose and fundamental law of any unit of formally 
grouped agents. 



A stfatement passed by vote of a representative body 
in accordance with its constitution, which may re- 
quire, permit, or prohibit particular actions. 



A statement issued under delegated authority orig- 
ination in and for the purpose of carrying out an 
enactment, which may require, permit, or prohibit 
particular actions. 



A formally unenforceable statement intended to sug- 
gest desirable conduct. 



An act conducted with no control other than the 
actor's judgment, within a decision zone which may 
be bounded by constitutions, enactments, regula- 
tions, or any combination thereof. 



A lefially binding agreement to do. or refrain from 



doing, a particular act. 



The body of principles and rules relating to the 
government and the security of persons and property 
which are not enactments and which derive their au- 
thority solely from usage over time and judgments 
of the courts upholding them. 



The statement of a judicial or quasi judicial body, 
often ascertaining -the existence of specific facts, 
issued under the authority of a constitution, an 
enactment, or both, and requiring, permitting, or 
prohibiting particular actions. 



The outcome of a -popular vote, at whatever level, 
however conducted, that leads to legally binding 
rule. 
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• What is the source of policy? Who (i.e., what agents 
in the configuration) could be the formal proximate 
author of the policy? 

• Who are the enforcing agents (if any) and what enforce- 
ment mechanisms (if any) are provided? 

• What might be the target of a given policy? To what 
purposive activities, agents, resources, and/or other 
policies might a policy apply? 

• To what superordinate policies is a known related? 

Figure .7 shows the basic structure and the general categories of the 
policy locating taxonomy. 

The following comprised a locational descriptors master list: 

• Federal 

• State (by name) . 

• Region* 

• County 

• District 

• School 

• Learning setting (e.g., classroom) 

• Nongovernmental 

Complementing the locational descriptors list is a parallel role 
descriptors master list. 

• Legislative Section 

- Legislatures 

• Judicial Section 

- Courts (by name if appropriate) 

- Attorneys General 



. ■ 

Regions are bounded in different ways for different KPU purposes. For 
example, there are 17 Boards of Cooperative Services in Colorado; each 
has its own name and serves a specific geographical region. On the 
other hand, a different set of regions is used by the Colorado Depart- 
ment of Education for assigning responsibility to state department 
actors who consult with and assist local and regional education agencies 
within that larger . region. 
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1.0 POLICY NAME (e.g., LEGAL REFERENCE) 



2.0 POLICY SOURCE 
2.1 AGENT 

— 2.11 LOCATIONAL DESCRIPTORS 

— 2.12 ROLE DESCRIPTORS 

3.0 POLICY ENFORCEMENT 

3.1 AGENT(S) 

— 3.11 LOCATIONAL DESCRIPTORS 

— 3.12 ROLE DESCRIPTORS 

3.2 ENFORCEMENT MECHANISMS 

4.0 POLICY TARGET (POLICY IMPLEMENTATION) 

4.1 AGENT(S) 

— 4.11 LOCATIONAL DESCRIPTORS 

— 4.12 ROLE DESCRIPTORS 

4.2 PURPOSIVE ACTIVITIES (BY NAME) 

--4.21 GENERAL KPU MANAGEMENT* 

— 4.22 RESEARCH 
--4.23 DEVELOPMENT 

— 4.24 LINKAGE 
--4.25 UTILIZATION 

4.3 RESOURCES (BY NAME) 

5.0 SUPERORDiNATE POLICY(S) 

5.1 OTHER POLICIES (BY NAME) 

*These five cattgorles are to be used only where activities can 
be clearly designated as one Or anothtr. 

Both Categories — Moro than Ont 

must be Used Category may bt Used 

FIGURE 7 STRUCTURE OF THE POLICY TAXONOMY 
80 
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Executive/Administrative 



- Federal education agency (FEA) 

Department of Health, Education, and Welfare (HEW) 
Assistant Secretary for Education (ASE) 

National Center for Education Statistics (NCES) 
U.S. Office of Education (USOE) 
National Institute of Education (NIE) 

-U.S. Executive (President, Executive Office) 

National Council on Educational Research (NCER) 
' Office of Management and Budget (0MB) 

(Other federal agencies having KPU-related roles — 
by name) 

- State education agency (SEA) 

Secretary of Education 
State Board of Education (SBE) 
Chief State School Officer (CSSO) 
State Department of Education (SDE) 

- Governor and executive offices 

(Other state agencies having KPU- related roles — 
by name) 

- Regional education agencies (administrative and service) 

- County Departments of Education 

- Local education agency (LEA) 

Local Board of Education (LBE) 

School District Administration (superintendent and 

assistants) 
iSchool Administration (principal) 

Department Administration (department head) 

Distrifct/School KPU specialists 

- Foraal linkage agents 

KPU information, centers or systems 
Teache rs cen te rs 

Education extension agents (EEA) 

KPU personnel training institutions (schools of education) 

- Formally established KPU advisory groups 

- Formally established accreditation groups 
Advocacy Section 

Interest groups and ad hoc groups 
Other and Nongovernmental Section 

- Professional associations and consortia 

- Professional media 

- Research and testing institutions 
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- Material producing institutions (including publishing 
houses) 

- Academic institutions (colleges and universities) 

- Education research and development laboratories and 
centers 

- Philanthropic foundations 

- Teachers 

- Students 



Summarizing Policies Using the Two Taxonomies 

To provide a ready reference to the main policies in any topic, we 
combined the major features of the two taxonomies. Under five headings, 
a tabular listing at the end of each topic summarizes each of the major 
policies that are most visibly important to the governance of agents, 
activities, and so forth in that particular configuration. The five 
headings are: 

• Policy name: the formal title of a policy (e.g., U.S. Constitu- 
tion, Elementary and Secondary Education Act of 1965, Board of 
Cooperative Services Bylaws, and so forth). 

• Reference citation: the legal citation (if any) for the policy 
(e.g.. Amendments Ninth and Tenth to the U.S. Constitution, 
Article 40 of the New York State Education Code, and so forth) 
or the issuing body if no formal citation is possible (e.g.. 
Office of Curriculum Development of the School District df 
Philadelphia, and so forth). 

• Type of policy and process of generation: respectively, the are 
the same categories defined in Table 9 for type of law and type 
of policy, 

• Policy source: The location and role name of the issuing body 
as per the locational and role descriptors master lists above, 

• Explanation of significance to the topic at hand: a summairy of 
policy targets, including the major agents, activities, and 
resources governed by the policy being described. 

The taxonomy completes the paradigm for describing and summarizing 
KPU governance. 



A Method for Mastering the Analytic Framework as a Set of Skills 



While the preceding presentation concerned what the analytic frame- 
work is and how it came to be, the goal of this section is to present a 
strategy to make the framework communicable as a set of skills. 

A Protocol of Procedures for Using the Framework 

To maintain continuity among our staff who conducted specific case 
studies, and to ensure that the framework would be useful to others (e.g., 
at NIE) , we developed a set of explicit procedures for using the analytic 
framework. Two of the case studies were mounted near the end of the 
formation of the analytic framework and as a result best show how it may 
be used in practice. These are Case Studies I (dealing with NIE allocations 
policy) and IX (dealing with the pilot state dissemination experiment in- 
volving educational extension agents). 

The four main steps of the protocol are schematically shown in 
Figure 8. Like the scientific method, using the framework for concept 
formation about the KPU infrastructure is part technique and part art. 

Step 1 ; Select and write the purpose(s) of the analysis 
or inquiry for which the analytical framework is to be 
used in as concrete a manner as possible. (Some different 
kinds of purposes are discussed on pages 40 and 41.) The 
statement of purpose is crucial as it sets up the heuristic 
decision rules for using the framework; it must be refined, 
however, as subsequent steps are taken, 

Selecting a purpos'e^ of analysis for the analytic framework is similar 
to selecting a new line of investigation in the physical sciences. It 'is 
largely a matter of art, intuition, and the curiosity or felt needs of the 
investigator. 

Step 2 ; Tentatively identify the focus of analysis, 
for example, a given agent, policy, activity, resource, - 
problem, or issue — the target of the first step above — 
and the field of analysis, for example, the set of agents, 
activities, and/or policies that will be considered as 
the bounded context within which analysis will be done. 
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Setting Goafs for Analysis 



STEP 1 

Select and Write 
the Purpose(s) of 
Analysis 

1 



Heart of Analysis 




Conclusions and Communicating Results 



STEP 5 

Complete System Maps, 
Highlighting Key 
interrelationships as 
Final Response to 
Purposes of Analysis 



Analysis Generally Proceeds Along this Line 

Reflection and Reiteration Follow These Lines as the Analysis is Refined 



FIGURE 8 OUTLINE OF THE ANALYTIC APPROACH 
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Selecting a focus of analysis is analogous to formulating a hypothesis 
in the physical sciences. It is the prudent generation of a rule for sort- 
ing matters of central concern from their surroundings. 

Step 3 : Identify the elements, that is, the agents, 
policies, resources, and activities making up the 
focus of analysis; relate them to other elements in the 
field to establish tentative relationships between and 
among them, for example, the agents responsible for 
activities through policy setting, enforcing, and imple- 
menting; and sequencing of activities in time. 

We found investigative reporting and searching policy archives the 
most useful methods for carrying out Step 3. 

In each case, the underlying strategy is to start with some basic and 

known component in the configuration and trace outward the connections and 

other components of the configuration. Invaluable tools in this process 

were the two taxonomies and a guide to legal archives we developed.' The 

most serious limits to this process are the limits of codification of local 

f 

school policy. However, this is rapidly changing in some localities. 

Step 4 ; After making this "first cut" at the signifi- 
cant elements, construct configurations that describe the 
system and how it works. Guide and test the adequacy of 
these configurations against (a) the purpose(s) of 
analysis (return to Step 1 and clarify the purpose if 
necessary); (b) the empirical evidence (using KPU liter- 
ature, legal and policy archives, interviews, and the 
like); and (c) the analyst's tentative image of the 
system (part of which is the underlying theory or model 
which the analyst thinks is appropriate). 

At this point in time, Step 4 requires more art than science. Mapping 
the connections and elements uncovered in Step 3 requires a great deal of 
judgment. However, a. variety of organizing techniques do exist which help 
in this task. Some of these are discussed in our interim report. 



This guide is presented in the last section of the bibliography in Sec- 
tion V of this volume. 

We recommend that NIE mount a study of the rapid growth in services to 
lea's to help codify their policy and keep them abreast of education law 
(see Section IV, No. 26-C). 
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Step 5 : Finalize significant portions of configuration 
maps according to the initial purpose of the study, high- 
lighting interrelationships between elements so that con- 
clusions are clearly drawn and justified as the final re- 
sponse to the important data that were collected in previous 
steps • 

This is the stage in which the analyst assesses what he has accom- 
plished and decides next steps (e.g., do more data collection, reconfigure 
the data, draw out new implications, or stop). 

Together, the protocol and the framework provide a systematic approac 
through which to inquire into any part of the ICPU system. What role they 
might play in support of a monitoring program will depend to a large ex- 
tent on the characteristics that are designed into the program. 

Our recommendations, based on what we learned in this study, are pre- 
sented in the following section. 
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IV DISCUSSION OF RECOMMENDATIONS 

t, — 

To sensibly present our recoinmendations we first review NIE*s objec- 
tives for the monitoring program and then discuss several issues we be- 
lieve should be addressed by NIE before any serious design work for an 
operational program is initiated. 

Objectives of the Monitoring Program 

The principal objectives of the monitoring system to be developed 
by NIE according to NIE's position paper^'Building Capacity for Renewal 
and Reform," and the Request for Proposal (RFP) leading to this study 
are : 

1. To achieve an understanding of the process of knowledge 
production and utilization in education. -This objective 
is fundamental to the success of other objectives of the 
Office of R&D Resources, and indeed, to the achievement 

of all NIE objectives. • ' 

2. To provide the data base needed for policy analysis. 
Such a data base would (1) provide an early warning sys- 
tem to identify problems requiring policy initiatives, 
(2) assist in weighing policy alternatives by supplying 
data on the context of decision-making and the predictable 
impact of policy alternatives, and (3) provide feedback on 
the consequences of policy initiatives. 

3. To provide for public accountability. The establishment 
of a system for monitoring KPU will provide a "feedback 
loop" permitting judgments about the impact of NIE ini- 
tiatives-- those des igned Jb^oth to_imp^r_o.ve__the--hatio — — - 

capability "f or conducting educational research and devel- 
opment and to change educational practice direct ly. 

In the process of rationalizing this program initiative, the authors 
of "Building Capacity for Renewal and Reform'' further noted: 



87 



Before we can determine how best to intervene to improve the 
system^ we need to develop a much better understanding of sys- 
tem dynamics.... To understand the KPU process, .. .we need to 
know a great deal more about the system indicators.... The 
concept of ^'monitoring" is borrowed from the literature on 
social indicators. That literature was originally focused 
largely on the identification and measurement of outcomes at 
a macroscopic level. More recently it has come to emphasize 
the need to conceptualize models of society or significant 
social subsystems and to use the models to identify the vari- 
ables in all parts of the system. As they are concerned with 
the dynamic interaction between model elements and the measure- 
ment and understanding of change, the indicators must be time- 
series. Once the interrelationships in the model have been 
established empirically, monitoring change in the variables 
becomes a means of anticipating change in other parts of the 
system. As development of such a model is a very long-term 
goal, it must be approached through a process of successive 
approximations. Even so, a beginning must be made. 

Task Force on Resources Planning and Analysis 
(1973), pp. 65 ff. 

Underlying Issues 

While the objectives discussed above are appropriately general, given 
that they formed the initial statement of a policy position, they must be 
translated into more detailed specifications before the design of an oper- 
ational monitoring program will be feasible. And, for this translation 
process to take place, there are (at least) four issues that need to be 
addressed by NIE: 

1, The degree to which NIE's monitoring program will be 

based on any given conceptualization or paradigm as op- 
posed to being based on a deliberate or haphazard mix- 

J:ure of conceptual- approaches* 

2, The degree to which NIE will try to rigorously articulate, 
(i.e., codesign and coordinate) its monitoring program 
and its other governance functions, 

3, The degree to which new knowledge will be cdnceptually 
limited to include only that which results from processes 
and/or products of the institutionalized KPU system, 
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4. The degree to which data needs and data collecting activi- 
ties of other agencies (particularly at the state level) 
will be explicitly considered in the design of the moni- 
toring program. 

A variety of alternative conceptions exists^ not only regarding the 
"proper" way to conceive of or understand such complex societal systems 
as KPU in education^ but also how they should be monitored and governed . 
The authors of NIE's position paper "Building Capacity for Renewal and 
Reform" recognized this fact when they recommended,, that a conference be 
held on alternative conceptualizations of the knowledge production and 
utilization system (Task Force on Resources Planning and Analysis, 1973, 
p. 69). 

The monitoring objectives specified in NIE's position are representa- 
tive of an essentially systems-oriented conceptualization that leads to 
the notion of an "external KPU monitoring system" to be developed by NIE. 
A contrasting conceptualization that illustrates how different conceptual 
paradigms lead to different recommendations for both a monitoring system 
and KPU governance is provided by Egon Cuba and David Clark's (1974) es- 
say, "The Conf igurational Perspective: A Challenge to the Systems View 
of Educational Knowledge Production and Utilization," in which they vig- 
orously attack NIE's position in "Building Capacity f or_Renewal_and_JR_ej=__ 
form."^ As we understand Cuba and Clark's position, they would have the 
function of "monitoring" be accomplished more through informal communica- 
tions between influential actors than through the formal reporting of 



A critical discussion of this essay, based on our study findings, is 
presented in Appendix B. We agree with Cuba and Clark's characteriza- 
tion of the nature of the KPU infrastructure in education and with their, 
recognition of the necessity to use "configurations" as an organizing 
rubric for further inquiry but disagree with their use of the term "sys- 
tem" and their suggestion that the proper NIE response to a constituency 
that is a "non- system" is to delegate much of its decision- making re- 
sponsibility to that constituency. 



needs, problems, plans, and expenditures, and more by negotiated articu- 
lation of programs between local, state, and federal tiers of government 
than through any "external" approach that vjould address KPU as a v?hole. 

These tv?o contending orientations regarding hov? KPU in education 
should be conceived and monitored are representative of a still larger 
set of contending "paradigms" v?ith vjhich to understand and influence com- 
plex institutional behavior. Table 10, dravjn from our Interim Report 
(Markley, 1974) classifies a variety of organizational conceptions found 
in the literature into three broad categories that form a.' multidimensional 
scalar. At one extreme of this dimension \^e find that it is unfeasible 
(\^hether or not it is desirable) to develop a "system" of integrated, co- 
herent, and f or\^ard- looking metapolicy, policy, strategy, and tactics. 
If such a vie\^ \^ere taken by NIE in its attempts to be "more vigorous in 
providing leadership for the direction and refinement of the nation's 
education research and development activities," and to develop a program 
for "monitoring and improving the Educational R&D system" (NCER Resolution 
No. 091875-19), the development of a monitoring program could take place 
without much regard for the overall leadership/governance posture of the 
Institute. 

At the other extreme is the view that the various levels of policy- 
making should be well integrated and coherent. Holders of such a view 
would likely assert that policy development should be based on research, 
planning, monitoring, and evaluation that is as rigorous, systematic, 
and well articulated as possible. - ^ 

A related conceptual definition yet to be made, which has enormous 
implications for the design of a monitoring system, concerns knowledge 
itself--the "K" in KPU. For purposes of conducting this study, we assumed 
that the knowledge in KPU was new knowledge (new to public education or 
to someone in the EKPU system); while it is clear the NlE*s jurisdiction 
covers all of KPU in education, we tried to restrict our focus to what 
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we referred to in shorthand as "EKPU(G)^" that is^ the production and use 
of new knowledge about the education process and/or its governance. We 
made no assumption about but were bothered by the lack of clarity regard- 
ing whether KPU should refer only to the creation and use of new processes 
and products for and in education by the institutionalized part of the KPU 
infrastructure^ or whether it should include new knowledge that is produced 
and used without ever having been'"blessed by the R&D establishment.*' 

In our preliminary survey of various agents in KPU we encountered 
teachers and administrators who stated that as much as 507o of the new ideas 
and techniques they try and use were invented by practitioners and passed 
on through informal face- to- face channels. Additionally^ in Case Topic V 
we found that one of the most significant obstacles to effective marketing 
of Minicourses for teacher inservice instruction was the lack of an identi- 
fiable agent at the local level with authority or responsibility to seek 
or try or purchase new innovations such as the Minicourses, Thus^ on the 
one hand^ both the criterion of realism and the congressional mandate to 
do a better job in addressing local needs argues for defining KPU in the 
broadest of terms^ thereby continuing the process of conceptual expansion 
that led in the first place from "R&D" to "KPU." On the other hand^ it 
would appear that there is almost no way that a monitoring system based 
on the social indicator conception could work within this definition. 

A Matrix of Concerns 

In keeping with NIE's initial request that this project emphasize 
. the investigation of discrete types of "regulators" in KPU, we were asked 
to conclude this exploratory study by making recommendations regarding: 

• Kinds of information on regulators which should be col- 



lected on a periodic basis as part of the program for moni- 
toring KPU. 

Implications for the design of the quantitative data base 
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indicators which should be monitored to determine the 
effects of important regulators. 

• Special research studies which are needed to understand 
the operation and effects of specific policies or groups 
of policies in general and on specific sectors. 

These three recommendations^ together with NIE*s three general objec- 
tives for the monitoring program noted earlier, make a convenient matrix 
for use in ordering a discussion of recommendations. Table 11 is so 
structured and provides a reference key to the ensuing discussion and sum- 
marizes the recommendations that are made in each "cell'* of the matrix. 

We do not believe that NIE's monitoring program is likely to have the 
necessary resources to make all recommendations operational. Hence, these 
recommendations are offered more as illustrations of the ki^nds of insights 
that stem from our study than as firm agenda to be pursued. 

In our closing section we highlight the recommendations that we judge 
to be feasible and reasonable next steps warranting serious consideration 
and support. These are organized by time sequence rather than by substan- 
tive topic. 

Recommendations for Understanding the KPU System as a Whole (1) 
Indicators (1-A) 

We earlier reported our finding that it is unfeasible to develop a 
set of general systems "maps" of KPU in education unless one specifies in 
some detail what it is that one wants to understand about KPU. This^ 
coupled with our conclusion that even if one does specify a purpose in 
some detail, many parts of KPU cannot be well summarized by social indi- 
cator statistics, leads us to recommend that^ in general^ questions stated 

^More recommendations are offered in most of the ten case studies de- 
scribed in Volume II of this report. 

9.3' ■ . 
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very broadly (e.g.^ What kinds of information on regulators should be col- 
lected on a periodic basis as part of the program for monitoring KPU in 
order to pursue the objective of achieving an understanding of the process 
of KPU in education?) not be pursued as a way to derive specific require- 
ments of the monitoring program. 

Design Implications (1-B) 

Although some proponents of one or another paradigm that specifies 
how KPU should be conceptualized^ monitored^ and/or governed would insist 
that there is one "right" answer^ we find (as has so often been found in 
the history of science) that it is neither necessary nor otherwise de- 
sirable to choose between contending conceptual viewpoints in any thorough- 
going manner. Rather, we believe that the question regarding KPU paradigms 
for monitoring and governance both will and should end up being framed not 
in terms of which conceptualization is "right" in some general sense, but 
in terms of which is most appropriate for a given purpose. 

For instance, the' use of our analytic framework in its various de- 
velopmental "generations" (described earlier in this report) on the ten 
case topics we explored led us to the conclusion that although the "social 
indicator external' monitoring system" paradigm is not very useful if one 
wants to understand cause-and-ef f ect relationships or the dynamics of 
change in KPU (phenomena that are more influenced by informal than by 
formal factors), it is quite appropriate for displaying much of the needed 
information about program budgeting and fiscal control of KPU in education. 
This is a part of the KPU infrastructure that is quite strongly influenced-* 
and even controlled — by formal policies; hence, by virture of its grounding 
in the formal legal system, it is a formal system, even though the develop- 
ment of formal policies is seemingly a most disordered process from a mech- 
anistic/rational systems perspective. 
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We recoTnmend, therefore^ that the monitoring program not be based on 
any single paradigm, but should be "paradigm plural;" it should be resil- 
ient, and able to work with any emerging paradigm that might come to com- 
mand the allegiance of most workers in KPU, The political astuteness of 
such a path is attested to by Ward Mason's observation that "a wide di- 
versity of approach is inevitably to be found in a field like education 
which lacks accepted 'paradigms* or models of what is important to study 
and how to do it" (National Institute of Education, 1975, p. 31). 

Operationally, this means that the sense of all of our recommendations 
is in keeping with the title of Donald Michael's recent (1973) book. On 
Learning to Plan--and Planning to Learn rather than with any "laundry list" 
of desirables that might be used for development of a monitoring program. 
In part, because of these facts, we finally were forced in our attempts to 
devise a feasible analytic framework for NIE's monitoring system, to create 
a system mapping analogue of the scientific method itself, together with the 
beginnings of a series of conceptual elements with which to use the method. 

Special Research Studies (1-C) 

Because of the difficulty we experienced finding taxonomic categories 
for use in a general analytical framework for KPU in education, and be- 
cause we found that it is not feasible to classify or map given agents, 
policies, Resources, or activities in highly aggregated fashion as a way 
to "understand" KPU, one of the essential conclusions we came to as a 
result of our study is that Cuba and Clark (1974) may well be right in 
their contention that the notion of configuration is a proper organizing 
rubric through which to seek understanding of KPU in education. 

The term "conf igurational" was chosen to describe the view 
adjectivally because it^ (1) connotes a conformation of ele- 
ments that exist in a definable territory; (2) assumes that the 
elements are (a) specifiable, and (b) relevant to one another; 
and (3) implies that the interaction of the parts is more than 
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the sum of the parts, as, for example, conf igurationism in 
Gestalt psychology. The term also implies that there is no 
direct analog available which can simply be chosen and used 
as ^ model for this particular cunf ifiuration of organizations 
as they relate to one another in terms of knowledge produc - 
tion and utilization . 

Cuba and Clark (1974), p, 30 ff (emphasis added) 

We recommend .that NIE support a study that would investigate the 
utility of this conception as an organizing principle for the monitoring 
program. To do so would be a direct extension of the work begun in our 
study, which developed a workable (and we believe satisfactory) taxonomy 
for classifying formal policies in KPU, The new study could serve as a 
first-approximation* feasibility of Cuba and Clark's contentions vis-a-vis 
monitoring and governance of KPU, in part by developing a satisfactory 
taxonomy of configuration types. 

A second special study that the Institute might sponsor is an investi 
gation of multiorganizational coordination in KPU. We. are frankly in 
doubt, at this point, about the kind of in-depth study that would be most 
worthwhile but agree with Sieber (National Institute of Education, 1975) 
that coordination and continuity of programs and policies comprise two 
very significant systemic problems afflicting the KPU "system" in educa- 
tion. A literature review might be a good starting point, however. In 
1972, when writing one of the original planning documents for NIE, we 
noted that: 

The literature relating to the state-of-the-art of multiorga- 
nizational coordination is very fragmented, has not become 
summarized and analyzed, and appears in large part to exist 
in the form of unpublished government memoranda or similar 
reports, 

Markley et al, (1972), p. III-35 

To our knowledge, such a literature review has still not been done 
and would comprise an excellent special study from which to infer monitor- 
ing implications, * 
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We could suggest a variety of other special studies but^ given that 
NIE has since its original planning days been besieged with more good 
ideas than it has money to consider^ we will not suggest more. 

Recommendations for a Data Base for Policy Analysis (2) 

NIE's objectives^ and consequently our discussion of recommendations 
regarding the data base needed for policy analysis is divided into three 
broad areas of concern: (a) early warning; (b) context illumination and 
impact prediction; and (c) feedback. 

Indicators for Early Warning (2a-A)- 

It is clear that the specification of particular types of information 
that should be collected to provide an early warning capability to identify 
problems requiring policy initiatives should not be done in the abstract 
but should depend to a large degree on the design of the monitoring pro- 
gram and the degree to which other "future-oriented" analysis activities 
(such as those recommended as special research studies below) are con- 
ducted. 

Given the recent evidence that educational lobbying has entered the 
**big time" ( Wall Street Journal , October 21, 1974) and NIE's own alloca- 
tions have recently been strongly affected by lobbying efforts (as noted 
in Case Topic I), it would appear that formal lobbying activities and re- 
lated political influence processes in education comprise one type of 
"regulator" which NIE might well immediately start to monitor; At least 
one periodical (The Congressional Quarterly's Weekly Report ) already 
exists which reports on lobby activities on major educational issues fac- 
ing the Congress, and we suggest that this and similar sources of infor- 
mation be monitored and made part of whatever type of early warning system 
NIE might ultimately devise and implement. Additionally, the whole field 
of KPU would be beneficially informed if subsequent editions of NIE's 
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Databook carried the names of all registered lobby groups that seek to 
influence legislation in education and/or KPU, the substantive agenda 
enunciated as having a special priority for each such organization^ and 
the budget expended by each for its lobby activities. 

Design Implications for Early Warning (2a-B) 

We list below various "early warning" events identified by SRI about 
two years ago. We include the list here as an illustration of how this 
type of information can be used to provide early warning on emerging edu- 
cational trend issues. Because this type of information cannot readily 
be synthesized from quantitative data bases, it is necessary to design 
into the monitoring program one or more "pattern recognition" procedures 
if this type of information is desired. 



For instance, the context of education at present (and in the foresee- 
able medium-range future) is one of fiscal tightening and declining en- 
rollment, thereby bringing a general contraction of discretionary (^e.g. , 
KPU- related) programs and personnel at the LEA level. With an "aging" 
teacher population caused by current seniority policies, the flow of new 
knowledge to operational classrooms from teacher training institutions 
and expensive in-service programs is likely to be much less significant 
than in the past, (The importance of this observation is indicated by 
the finding that "comparatively small amounts are usually spent onlip- 
grading teacher competencies when innovations are introduced, ,. , There 
is a tendency, when innovations are introduced, to hire additional per- 
sonnel already possessing new skills rather than to retrain the current 
staff and reassign them to new functions" (Hyer and McClure, 1973), 
Therefore, materials and processes that can be readily used by existing 
personnel and that either have a low^ initial cost or can be amortized 
in various ways appear to be most significant for current policy consid- 
eration. Conventional textbooks, transportable and reusable in-service 
packages, and utilization ventures entailing cooperative consortia of 
schools appear to meet these requirements. Hence, they are judged to be 
particularly significant activities to plan for and to monitor, other 
things being equal, 

99 



Highly probable/almost certain 

- Declining enrollment^ unused facilities and staff lay- 
offs--unless structure/function of public education is 
reorganized and expanded. 

- Financial squeeze^ nonpassage of bond issues and tax 
overrides^ rising costs but declining income. 

- Increasing unionism--possibly extending through admin- 
istrators and university professors^ with public school 
collective bargaining virtually accomplished on a state- 
wide basis. 

- Emphasis on software (programming^ conceptual aspects) 
rather than hardware (equipment) in educational technology. 

- Broadened credit certification of students (GED high school 
equivalence test^ credit for outside experience^ etc.). 

Reasonably probable 

- Changing credentialing/ certification of professionals 
(multidisciplinary community consortia for credentialing 
of noneducator experts requirement of renewal of creden- 
tials; question of credentialing for publicly supported 
"nontraditional" education) . 

- Universal post-secondary educational opportunity (both 
collegiate and nontraditional modes of study; with the 
nontraditional modes increasing as the supply of college 
graduates exceeds the demand for their services) . 

- Recurring education [on again/off again full and part- 
time study^ K through D(eath)]. 

- Emphasis on '*out of school" experiences as an essential 
part of education; the school being a broker^ expediter 
and overseer to facilitate this process, 

- Conflict regarding '^stimulus control" (control by others) 
versus "respondent control" (control by self) in 'potent 
new socio- and psycho- technologies as used in public 
schools. 

Conceivable 

- Year around schooling 
more effective. 



to save money and make education 
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- Reversal of seniority aspect of tenure laws as applied 
to layoffs (due to lawsuits based on equal opportunity 
considerations)--quality/ef fectiveness being used in- 
stead. I 

- Educational vou^cher sy^stein^ allowing students/parents 
to choose publicair^private schools or other way of ac- 
complishing education. 

- Reversion to "3-Rs" emphasis in many public schools. 



Special Research Studies for Early Warning (2a-C) 

We suspect that for some- time to come NIE will not have the necessary 
resources to develop an adequate system to provide an early warning capa- 



_bi-l-i-ty--(-u'bj"ecTi?veTa)^ to provide data on the context of decision-making 
in KPU^ or to provide for feedback and public accountability in any wide- 
spread degree. We therefore recommend that NIE seriousl}'" consider develop- 
ing policies that would combine both governance and monitoring of KPU in 
ways that would tend to fulfill all of these objectives. As an example 
of how governance and monitoring might be so combined^ we note that a 
policy used by USOE in connection with their governance of the ESEA Ti- 
tle III Teacher Initiated Innovation Program discussed in our Case Topic 
VII appears to have valuable characteristics for NIE to consider. Es- 
sentially USOE imposed a requirement that each state wishing to participate 
in the program would have to prepare a state plan meeting certain proce- 
dural but few substantive requirements. This essentially allows a federal 
agency to monitor the felt needs and planned responses on a state-by-state 
basis without ^becoming overly involved in data collection. , USOE also made 
full funding contingent on receipt of an. acceptable plan with partial 
funding guaranteed even if the plan was not approved. By use of this 
policy^ USOE obtained a strong "leverage" effect on weaker SEAs^ through 
which it was successful in encouraging them to engage in a type of sys- 
tematic KPU governance activity not previously engaged in. 
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If NIE does wish to pursue the types of studies that would lead di- 
rectly toward these objectives^ however^ we suggest that our recommenda- 
tions regarding "Research on the Societal Context of Education" made during 
the preliminary NIE planning phase be consulted (cf, Markley et al.^ 1972), 

Indicators for Context Illumination and Impact 
Prediction (2b-A) 

We find no basis for recommending any specific indicators for fore- 
casting or estimating the predictable Impact of policy initiatives unless 
particular policies and particular types of impacts are first specified. 
(For example^ we recommend below an assessment of the impacts of the 
Buckley Amendment. ) 

A variety of indicators could be collected that would supply data on 
the context of decision-making in KPU, We recommend one that would help 
illuminate the degree to which balance and continuity-- two systemic proper- 
ties of critical importance to a healthy "ecology" in a complex living 
system--are displayed by fiscal allocation and procurement in KPU. 

In our Case Topic II (dealing with NIE procurement policy) we noted 
that NIE has an expressed preference for the competitive-contract mode of 
procurement. In Case Topic IV (dealing with policies influencing research 
performers) we learned of various ways in which excessive reliance on this 
mode by NIE produces counterproductive results (e.g,^ causing secrecy and 
proprietary posturing where an attitude of collegial openness once pre- 
vailed regarding one's exciting new hypotheses and creative insights about 
KPU), Also in our preliminary survey to identify important "regulators" 
in KPU^ universally the most commonly nominated influence was "money," 

Because of these findings and because we note that NCER has recently 
passed a resolution asking NIE to take a fresh look at its procurement 
policy^ we recommend that NIE monitor and publish in its Databook a cross- 
tabluation of its disbursements in various categories. As specific 
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categories to consider, we suggest the level of funding underlying its 
principal procuremenc modes (e.g., field- initiated grants)^ its principal 
substantive categories of support (e.g., dissemination, basic skills, 
equity)^ and the types of T^cipient..Xe^.-g.-j^ 
state agency). 

Design Implications for Context Illumination and 
Impact Prediction (2b-B) 

Again we express our finding that attempts to map (and, by inference, 
to monitor) the KPU infrastructure in terms that are general yet suffi- 
ciently specific to permit the drawing of detailed inferences about KPU 
dynamics and structural influences is not feasible. Therefore we recom- 
mend that **identif ication of key quantitative indicators*' be attempted 
only after NIE's policy analysis data needs have been Identified. We 
expect that a convenient way to portray this relationship would be to 
construct a matrix in which one dimension of classification would repre- 
sent the various purposes of analysis needing to be informed by informa- 
tion about the KPU infrastructure and the other representing the various 
configurations in KPU' whose infrastructure and characteristics would need 
to be monitored in different ways due to their different nature. 

Special Research Studies for Context Illumination 
and Impact Prediction (2b-C) 

Although a variety of studies could be recommended here, we will only 
provide illustrations of four different studies that we find important. 
One would look at program and policy codification efforts performed by 
others which might be useful parts of NIE's monitoring program. The sec- 
ond would direcfly identify policy analysis data needs that should be con- 
sidered when designing the monitoring program. The third would lead to an 
ability to estimate or forecast the predicted impact of policies on KPU^ 



and the fourth would involve an experiment with an innovative approach 
to program management at NIE. 

1. Using Program and Policy Codification Efforts 
Performed Elsewhere 

A logical next tep beyond the monitoring and reporting in cross- 
tabulated form of NIE and other KPU procurement activities could well in- 
volve the cataloging of federal KPU program obligations. This could 
involve extracting from such existing catalogs of educational program 
obligations as USOE's Catalog of Federal Education Assistance Programs 
and cataloging KPU assistance programs supported by nonfederal agencies 
as well. Such an effort would require such an extensive clarification 
in the definition of "KPU'* that it is not feasible at the present time, 
although a study of its feasibility might be undertaken. 

An effort that might lead to activities that can be implemented 
immediately would involve studies to determine the utility of policy codi- 
fication efforts currently being done either commercially or by other gov- 
ernment agencies in education. Companies such as the Commerce Clearing 
House and Prentice-Hall Publishers maintain reporting services on various 
aspects of the law. Officials at the Commerce Clearing House have ex- 
pressed an interest in the feasibility of extending its "CCH Reporter on 
Higher Education" efforts into other aspects of education. 

We found in researching Topic IX that a significant number of 
LEAs are beginning to contract with service agencies for services such as 
the codification of their policies and the monitoring of policies emanat- 
ing from higher levels of governance of which they must be aware. The 
Southern Carolina School Boards Association, for example, has been active 
in policy codification for various LEAs in the Southeastern states. This 
group is able to do an initial codification task in a typical LEA for be- 
twGcu $2500 and $ 5 000, with Lhe cusL of upd ating running about i^^Oii per 
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year. In California^ a publication entitled the California School Law 
Digest keeps the LEAs abreast of the judicial decisions affecting the 
operation of California schools. This digest is published monthly and 
costs $30 per year. 

We hypothesize that the general adoption of these services by 
the LEAs would greatly enhance NIE's policy-monitoring capability. In- 
stead of attempting to build an extensive policy information bank--ex- 
tremely expensive both in terms of initial development and updating--NIE 
policy analysts would need only to learn how to use the various codes and 
other services being produced at the local and state levels. For this 
reason^ we suggest that NIE investigate the various services available. 
By informing and encouraging the LEAs relative to the availability of the 
various legislative information services^ NIE might increase further the 
rate at which local education policies are becoming codified and hence 
available in a realistic sense for policy analysis. 

We note that the NIE research library (part of NIE's Educational 
Resources Division) is considered the principal federal library in the 
field of education and is already , collecting and cataloging various types 
of information on educational research and governance. With a small in- 
crease in its reference staff^ this facility might be appropriately used 
as an adjunct to the monitoring program staff, dealing primarily with 
highly codified policy and program information archives needed for a policy 
monitoring capability. 



2, Identify KPU Policy Analysis Data Needs 

To repeat, it is generally not feasible either to "map" the 
regulatory system structure of KPU in education or to "monitor" its system 
dynamics unless sufficiently specific purposes of mapping/monitoring are 

and dynamics should be highlighted. We therefore recommend that an early 
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task in the development of the monitoring program be a systematic identi-. 
fication of major policy analysis data needs that are likely to be felt 
by NIE and other significant policy setting bodies in KPU (principally 
SEAs and congressional staff) over the next two to four years. We mention 
the policy analysis needs of SEAs for two reasons: (1) the primacy of 
state governance in educational matters is the preferred policy of the 
United States as expressed by both the Constitution and by congressional 
legislation such as 20 USC 123a (Prohibition Against- Fede"r^1L~~Control of Edu 
cation); (2) with the recent philosophy of revenue sharing and other trends 
it appears that the state tier of government may be on the ascent in in- 
fluence relative to other levels of governance in the United States. After 
receiving federal support via ESEA Title V for almost a decade, many state 
agencies are now attempting to build effective bureaus of research, plan- 
ning, and evaluat ion--bureaus that are perhaps uniquely qualified to col- 
laborate with NIE in building an effective R6cD support system in education. 

Given the important policy analysis data needs of Congress, as 
well as the importance of NIE*s relationship to Congress, the data needs 
of congressional policy analysis agenda might be considered as well. 

3. Assess Probable Impacts of the Buckley Amendment 

An example of a policy analysis data need that we believe should 
have a high priority for NIE and that well represents the type of policy 
analysis requiring quantitative information about the KPU infrastructure 
is the recently enacted Buckley Amendment (PL 93-380, Section 438) was 
explored in Case Topic III. This amendment seeks, to protect the privacy 
of students by requiring, among other things, that researchers wishing 
to gain access to student records obtain permission of the parents of 
the students involved, and that they place in the student file a state- 
ment that the file was so used and for what purpose. Educational offi- 
cials who use such information for improvement of public school programs 
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and researchers were are "representatives of educational officials" con- 
ducting evaluations of federally sponsored educational programs are 
specifically exempted from the parental permission requirement. 

A formal analysis of the probable impacts of the Buckley Amend- 
ment would provide NIE officials and other KPU professionals with needed 
information with which to provide leadership in influencing Congress to 
change this legislation^ should this appear to be worthwhile. As discussed 
in Case Topic III^ such an analysis would require information about the 
KPU infrastructure that is not currently available in NIE's Databook, How 
to obtain this information represents an excellent case example of the 
trade-offs to be faced in deciding what types of information the monitoring 
program should provide. We recommend that NIE sponsor both a policy im- 
pact assessment of the Buckley Amendment and^ as part of this assessment, 
an appropriate survey of the KPU infrastructure to identify all types of 
agents and activities likely to be impacted, 

4. An Experiment with Innovative Program Management 

We wish to go considerably beyond the assigned scope of the re- 
commendations we were asked to make in this study and to suggest that ,NIE 
consider experimenting with an organizational innovation through which 
program management at NIE (and other federal agencies as well) might be 
significantly improved. 

Although we do not describe the rationale underlying KPU design 
nor some of the long-term advantages and disadvantages it might off^r, we 
note that the complexity of the KPU system is such that a sufficient level 
of understanding for adequate program management usually lies beyond the 
capability of any single administrator or any staff. We believe that no 
combination of monitoring and analysis that could feasibly be conducted 
will significantly change this situation given the usual ways in which 
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functional responsibilities 
the government. 



are divided in administrative agencies of 



We therefore recommend that NIE consider conducting one or more 
intramural experiments in which a project management team would replace 
the usual program officer-contracts officer team. In such a team approach 
the following types of personnel would be valuable--both to assist in 
project management and to fulfill many of the functions that now look un- 
feasible for a nonhuman '^monitoring system" to fulfill: 

• One or several substantive specialists who should know 
the current state of the art from the academic/ technical 
perspective. One or more might be nationally known 
experts who would serve the team in a consulting capac- 
ity^ thereby assuring technical adequacy. 

• A program historian who should know the past programs 
that have been mounted by various agencies in the topic 
area (and what became of them). This would assure a 
higher degree of coordination of programs supported by 
NIE with other past and present efforts. 

• A policy specialist who should know the significant 
policies and organizational structures that exist at 
the federal^ state^ and (to the extent possible) local 
level. This would assure that NIE programs are de- 
signed so that they would be more apt to interface prop- 
erly with organizational requirements in the field and • 
comply with significant policies throughout, 

• A combination social scientist / investigative reporter 
who should get to know "what's happening" and "how 
things actually work" in the field from an impression- 
istic/anecdotal perspective. This person would spend 
at least half time in the field talking to different 
KPU personnel. This would assure that NIE programs 
interface properly with the informal as well as the 
formal customs^ policies^ and constraints that actually 
influence KPU most strongly, 

• A contracts specialist who would act much as this type 
of agent does at the present time^ but who would ad- 
ditionally take a leadership role in seeking change in 
administrative requirements that make effective procure- 
ment difficult, 
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These specialists would work together with NIE management to de- 
sign, manage, and evaluate/ redesign NIE programs, and in so doing, might 
be able to alleviate shortcomings in the process that are usually not even 
generally detectable without some such approach. For instance, rapidly 
changing federal funding and program management policy in educational 
research over the past decade and the lack of postprogram audit and 
follow-up C^next stage^') programming often have contributed to the in- 
efficient and disheartening practice of supporting good programs which, 
when completed, simply "disappear"-- not because the programs themselves 
were deficient, but because of a lack of mechanisms to ensure continuity 
coordination and integration in KPU, 

Given that NIE is permitted to spend only 10% oZ its funds in- 
tramurally, one might argue that its intramural research should be oriented 
toward intramural topics. . Whether or not such an argument has merit, we 
view the above experiment as one that has exceptional significance, not 
only for NIE but for federal research program management generally. It 
should be noted, however, that the information necessary for the program 
historian and the policy specialist to work effectively has not been col- 
lected. Therefore, an initial effort on such an experiment might be to 
draw these types o*f information together. (The persons involved in the 
initial fact finding regarding program history and existing policies 
could, of course, become the specialists in the proposed experimental 
team, thereby using the conduct of the initial studies as a way to tool 
up for the new role. ) 
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Indicators for Feedback [2c(and 3)-A]* 

We recommend that no attempt be made to identify periodic data col- 
lection efforts until those policy analysis data needs having highest 
priority have been identified. 

Design Implications for Feedback [2c(and 3)-B] 

The design of a monitoring system having a capability "to provide 
feedback on the consequences of policy initiatives both those designed 
to improve the Nation^ s capability for conducting educational R&D and 
those designed to change the schools directly" is, we believe, clearly 
beyond the state of the art and will remain so for some years to come if 
this capability is interpreted at all literally or rigorously. There 
are a number of different ways to pursue this goal, however. 

Assuming that the study identifying and describing the important 
configurations in KPU has been done, it should be possible to do a peri- 
odic survey of problems, needs, and changes in KPU that are perceived 
by representative actors in each of the major configurations. By col- 
lating and comparing the responses as a function of configuration repre- 
sented (possibly using discriminant analysis or some related statistical 
technique), it might be possible to notice changing patterns in reported 
problems that would to some extent follow program initiatives undertaken 
by various agencies in KPU. 

Alternatively, depending on the positions that NIE ends up taking 
on the four issues noted at the beginning of this chapter, it might be 



Because of the substantive similarity between "feedback" and "accounta- 
bility," as these two terms were used in NIE' s objectives, we combine 
them for convenience in this discussion. 
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possible to develop feedback mechanisms that would depend on the planning 
and assessment activities of agents in various configurations throughout 
KPU. 

Still a third possibility would be to promote the existence of and 
to depend strongly on the perceptions of personnel whose professional 
responsibilities are primarily concerned with knowing what actually goes 
on in various parts of the KPU infrastructure, such as the "education 
extension agents'' studied in Case Topic IX, or the "social scientist/ 
investigative reporter" envisioned in the intramural experiment (2b-C) 
above^., ' .t 

Special Research Studies for Feedback [2c(and 3)-C] 

For reasons discussed above, we recommend that NIE place other moni- 
toring objectives above this one. Once its higher order priority concerns 
are being fulfilled, however, one or more studies might well be conducted 
that would investigate the feasibility of implementing such feedback moni- 
toring -mechanisms as were discussed in 2c(and 3)-B above. 

Recommendations for Sequential Action 

As was stated earlier, we would not seriously recommend that NIE pur- 
sue all of the possibilities expressed in the preceding section. In order 
to highlight those recommendations we believe are most appropriate for 
NIE's serious consideration and place them in a temporal perspective 
needed for realistic policy development, we discuss a few of "these recom- 
mendations in the sequence in which they could feasibly be pursued. The • 
recommendations so selected represent either issues' needing to be resolved 
before subsequent design steps can take place coherently, or monitoring 
actions that are currently needed and can take place independently of the 
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strategy NIE ultimately selects on which to base its monitoring 

Such recommendations would lead to an improved understanding 
is feasible to monitor_in general. 



Identify KPU Policy Analysis Data Needs 

At various points in our discussion we have sought to convey our 
finding, that the design of a monitoring program should be based to a Large 
extent on the actual data needs of the policy analysis activities it is 
to support. This is not to say^ however, that the monitoring program 
should include only those data collection activities that have immediate 
and direct use. Rather, a whole level of detail more specific than that 
set forth thus far regarding objectives and purposes for the monitoring 
program needs to be specified by NiE before proceeding much further, and 
the identification of KPU policy analysis data needs (both within NIE and 
elsewhere) should be one of the essential parts of this process. 

We do not envision this as a massive undertaking, but rather a rela- 
tively direct process of asking various individuals and groups known to 
do or planning to do important types of policy . analysis regarding KPU in 
education what analyses they envision doing over the next two to four 
years, and to infer the types of information about the KPU infrastructure 
(see Table 12) they would need to do more valid, informed, or defensible 
analyses. The individuals and groups might be sampled from within NIE only 
or from Congress, other federal educational agencies, state. educational 
agencies, and possibly from the professional community as well. We sus- 
pect that the sampling plan would depend to a large extent on the types 
of choices that NIE makes regarding the following issues that we discussed 
earlier: 

1. The degree to which NIE's monitoring program will be 

based on any given conceptualization or paradigm as op- 
posed to being based on a deliberate or haphazard mix- 
ture of conceptual approaches, 
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overall 
program, 
of what 



Table 12 

THE FOUR SAMPLE PURPOSES OF ANALYSIS 



1. Identify how a particular policy contributes to the completion of some higher level goal 
through investigations of: 

(a) Activity coordination : identify the policies and activities that serve to coordinate 
the target activity with other activities in service of larger KPU goals. 

(b) Resource accessibility : identify the adequacy of resources to carry out a particular 
policy. 

(c) Policy coordination : identify how a specific policy coordinates a set of policies to 
specify a given activity or to regulate some agent or resource, 

(d) Agent/activity impact assessment : list responsibilities and concerns carried by a set 
of agents or activities and estimate the impact of the target policy on the life of 
one or more of these agents or activities. 



2. Identify the elements of the policy through the investigation of: 

(a) Agent/activity identification : list all agents or activities having enforcement or 
compliance responsibility under a given policy. 

(b) Activity analysis : break the activity specified by the policy into its constituent 
stages and elements (policies, agents, and resources), and show how the elements in- 
teract as the activity is performed. 

(c) Policy/activity design: on the basis of existing knowledge, assess each stage of a 
proposed policy or activity in terms of requirements for proper functioning, - 

(d) Policy identification : list all significant policies . 

(1) . A given agent or activity is responsible to enforce, 

(2) A given agent or activity is responsible to comply with. 

3. . To identify the impacts of policy through: 

(a) Impact identification : map all the activities, agents, resources, or policies that 
are (or might be) significantly impacted by a given policy, 

(b) Agent/activity impact assessment : analyze the effects on a given agent or activity of 
the entire spectrum of policies acting on it, 

(c) Policy dynamics : identify other policies that aided or impinged on the developmental 
process, 

4. Describe policy simply and clearly through: 

(a) Identification of the policy (s), generated at the level of the aggregate agent, and 
specification of the stages of- the activity. 

(b) Identification of the various stages of the activity in order of occurrence, the par- 
ticular agent responsible, and the requisite resources, 

(c) Identification of other policies that affect these in their operation. If indicated, 
describe conflicting policy(s)« 
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2. The degree to which NIE will try to rigorously articulate 
(i.e., codesign and coordinate) its monitoring program and 
its other governance functions. 

3. The degree to which new knowledge will be conceptually 
limited to include only that which results from processes 
and/or products of the institutionalized KPU system. 

4. The degree to which data needs and data collecting activi- 
ties of other agencies (particularly at the state level) 
will be explicitly considered in the design of the moni- 
toring program. 

Address Underlying Issues 

Unless NIE should decide to choose the ^^least effort" alternative 
in each of the above four issues (i.e., haphazard mixture, no articula- 
tion, NIE data needs only, institutionalized KPU only), we strongly rec- 
ommend that NIE, in addition to holding a conference on alternative 
conceptualizations, find some way of addressing and making tentative 
decisions regarding issues such as (but not limited to) the above four. 
Given the council's resolution on seeking input from outside the Insti- 
tute, one possibility would be to hold, in addition to the conference on 
alternative conceptualizations, a workshop in which invited participants 
would join NIE staff in addressing^ these issues which have profound long- 
range implications regarding how KPU is to be conceptualized, understood, 
and governed. 

Monitor KPU Obligations and Procurement Activities 

We suggested above that the monitoring of KPU obligations and pro- 
curement levels in various categories would make an ideal addition to the 
Databook. By preparing cross- tabulated levels of support for KPU activi- 
ties across time in such categories as procurement mode, substantive topic 
addressed, and type of recipient, several important system indicators 
could be developed. One indicator could deal with the balance of support 
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and one with continuity of support--both for particular configurations 
and for the KPU system overall. 

As an alternative to the "social indicator" approach we also recom- 
mend that NIE explore various ways of using the program plans and reports 
of SEAs, one of which we described above in 2a-B, 

Assess the Probable Impact of the Buckley Amendment 

As stated earlier, assessing the likely impacts of the Buckley Amend- 
ment (PL 93-380, Section 438) is a task that (a) currently needs to be 
done; (b) represents a type of analysis believed to be increasingly nec- 
essary in the future; and (c) requires information about the KPU infra- 
structure that is not currently available in the Databook, Essentially, 
as we showed in Case Topic III, such a policy impact assessment, if at 
all rigorously done, would require at a minimum a listing of all actual 
research activities in KPU that would have been subject to the amendment 
as a way of estimating how many and what types of activities are likely 
to be impacted in the future. A more ambitious assessment would require 
the use of some method such as we developed in this study to trace out 
significant cause-and-ef f ect relationships through the system. Because 
of the way in which the law is written, this listing must involve consid- 
erations of, for example, the purpose for which the research was done, 
under whose auspices the research was done, and whether evaluation of 
federal educational programs was involved. 

Although we essentially concluded that the data needed to assess the 
impact of the Buckley Amendment would be best obtained by doing a special 
survey--as opposed to assuming that the monitoring program would or should 
ever attempt to make information about the infrastructure available at 
such a fine level of detail--this task is perhaps a good example of a 
policy analysis data need to use in exploring various trade-offs to be 
faced in deciding what types of information the monitoring .program should 
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or should not attempt to provide. Additionally, it offers NIE personnel 
a concrete instance for use in developing internal policy through which 
to articulate monitoring functions, policy analysis functions, and gov- 
ernance functions. Assuming, for example, that NIE decided to conduct 
one or more special surveys as Phase One of a policy impact assessment, 
the question arises whether the supervision of such surveys should be 
vested in the Monitoring Program Office or within a substantive office 
of the Institute. 

Identify and Describe Configurations in KPU 

We noted previously that a study to investigate the utility of the 
notion of configuration as an organizing rubric for understanding KPU 
would continue the efforts we began in this study to develop an analytical 
framework for understanding KPU and \^ould represent the next logical step 
in its development. 

Cuba and Clark suggested six categories as an appropriate starting 
place for such an effort (see Table 13), and then went on to suggest 
several questions and ways of answering those questions that should be 
employed (Cuba and Clark, 1974, pp. 31-32). A published enumeration of 
the membership in -each category of this essentially agent- centered tax- 
onomy would be a valuable contribution of a monitoring program. Even 
more valuable, however, would be one or more representations of how these 
agents tend to form into various configurations around common goals re- 
lated to improvement- orient ed-xhange—i-n-eduGart-i-on^ 

Consistent with what has been said above, however, we caution that, 
to ensure a categorization of configurations likely to be useful to the 
needs of policy analysis in KPU generally, this study should not be under- 
taken until the identification of KPU policy analysis data needs has been 
conducted. Thus, although the results of this task would greatly assist 
in the task of assessing the impact of such policies as the Buckley 

116 



H 



H 



o 
u 



25 
O 
M 

a 
w 

H 



:3 



CO 
23 
O 
M 
H 

:3 

H 
M 
H - 
CO 
?5 



O 

§ 

U 
CO 

23 
O 
M 





















e 






t 
















13 






1 


1 












w 










1 








13 


•H 










1 




1 


01 






c 


U 












rH 










w 


1 


rH 






JJ 










•rl 






C 






c; 


3 


M 








U 


0 










•H 


X 


U3 






•H 






Qi 


H 


U 




0 


00 






00 


(X 


0 








•H 


0 


c 








M 


(U 


0 




f-t 


rH 






C 




Cu 


• 


CJ 


•rl 






CO 












3 










(U 


(U 




u 




0 


0 


en 




•H 










Vi 








0 










sc 


<u 






•H 


JJ 
CJ 


ed 


cu 




*4-l 


a. 


ne 




rH 


_f 
P 


le 


JJ 

3 


?>> 

rH 


as 


as 




al 


JJ 


Qi 
> 








/- — \ 




00 






JJ 


CO 




rH 




13 




Qi 




13 


CJ 


,Ci 


0 


•H 




•H 




3 


w 


•H 








W 


00 








c 


U 


CO 


CO 




C 


• 


> 




CJ 


JJ 


•H 


?>> 


M 


o 


CJ 


1 


13 


•H 


JJ 










c 


CO 






(U 


CO 


fC 


c 




CO 


00 


'H 




C 


C/3 


cn 




CO 


M 


C 


>H 


•H 


W 


C 










•H 












W 


0 






• 


JJ 




00 


?^ 


C 


13 


B 


CO 




CJ 


> 


CO 


CO 








nj 


■U 


CJ 


• 




(U 


CJ 




•H 




W 




CJ 




•H 


Vi 


0 


C 


•rl 




00 




•r-l 


rQ 


JJ 








0 


CO 




w 










JJ 




(U 




01 




W 




Cu 


CO 


M 


0 


CO 


B 


13 




OT 








00 


u 


00 


fH 




JJ 






CO 




W 


•< 


H-i 




W 


rH 


Vi 




Cu 


fC 




CJ 




rH 












(U 


c 


CO 






c 


u 


M 




W 


JJ 


^-^ 




CO 


CO 


Qi 


C 




3 


rH 


6 


•H 


CO 


3 












•H 


•H 




w 


u 


c 


(U 




CO 


w 


Qi 






c 


U 


Qi 


Q) 




CO 






c 


OT 








cn 


0 


JJ 


>-» 






(U 


(U 


CL, 




rH 


Qi 






Qi 


0 




M 






3 




0 


0 










c 




CJ 


(U 




§ 


CJ 


3 


0 




CJ 


U 


U 




g 


•H 


Qi 


13 






13 


•H 


JJ 


•H 










0 


•> 


(U 


jJ 




•H 


c 


rH 








Qi 


0 




0 


JJ 


U 


r-< 


■*"* 


JJ 


•H 






JJ 


I-H 






c 


•H 






CO 




JJ 


0 








CJ 


JJ 








CO 


CO 


•H 


0 


0 




M 


rH 


CO 


CO 






0 


iJ 


00 




e 




(U 


CJ 


c 


0 




•H 


C 


?>> 






CJ 






C 


c 


•H 


CJ 


c; 


c 


CJ 






•H 


to 


• 








g 




•H 






M 


•H 


u 




rC 


3 


w 


CJ 






13 




rH 




• H 






JJ 












0 


rH 




(U 




(U 




c 




JJ 


13 


(U 




?>> 




C 


JJ 


rH 


M 


JJ 






CO 


(U 






CO 




V) 


CO 


r-i 


a. 




C 


M 


(U 




•iH 


CJ 


'H 


00 


CO 


rH 


•H 


CJ 


CO 


0 


•rl 






M 


Oh 




V) 


c 






•H 


CO 


?>> 




(U 


0 


•H 




3 




CJ 


c 


e 


rH 




00 


M 




rH 






1-1 


0 




(U 


0 






CJ 


U 


JJ 




00 




U 






C 


C 


•H 




CO 


c; 


0 


CO 




0 






U 




(U 


•ft 


•H 




V) 


(U 


•H 








(U 


•H 




w 


CO 


Qi 


JJ 


M 


u 


fC 




a 


rH 


Cu 






(U 


>> 


iJ 


CJ 


JJ 




w 


a. 


JJ 






rH 


W 


(4-1 




(U 


CJ 


00 


CO 


0 


Qi 


JJ 






CO 








JJ 




c 


c 


U 




CO 


w 


•H 






CO 


0 


(4-1 




•H 


•H 


CO 


U 




c 




c 




c 








U 




a; 


Q) 






rH 




rH 


^a 




u 


fr 

H 


Qi 




CJ 


M 




3 


?>> 


Qi 


(4-1 


•rl 




0 


• 






CO 


p 


•H 


00 


s 




CJ 


CO 


0 


•H 


• 








C 


o; 


fC 


13 


CO 




0 


0 


c; 


•rl 


CO 






>-( 


•H 


u 


CO 


JJ 








(X 


C/3 


>> 


> 


3 


00 




(U 


1 


u 


Qi 


E 






•r-) 




00 


• 






nj 




0 




cn 




CJ 


?^ 






CJ 




(X 


c 




00 




3 












•H 


Qi 


c; 






rn 


Q* 




r-i 


c 




•H 








c 


w 




•H 




CO 




w 


(4-1 


0 




c ■ 


0 




U 








u 






CO 


•H 










(U 


(U 




C 


JJ 






a; 




0 




JJ 


CO 










Qi 






•s 


fii 


c 






(U 




00 


00 


00 


E 


o 


CO 




00 


fC 


(4-1 




3 


•H 




3 






OT 


u 






c 


2 


0 


00 




C 


(U 


• 


13 


CO 


0 




U 




c 


JJ 


0 






> 


c 




rH 


Qi 


4J 


CO 






0 




•H 


c 




(U 


,c 


(U 


(U 






E 


Qi 




•H 






W 




•H 


•ri 


w 


rH 


JJ 


OT 


(4H 








a; 


JJ 


•H 




00 


w 




rH 


(U 




0 


Cu 




JJ 


c 


^ 


CO 


cn 


JJ 






CO 


CO 


Qi 


rH 






i 




CO 


Q* 






•H 




3 


rC 




U 


0 


Qi 


CO 


•H 


r-l 


JJ 


M 


CJ 


W 


0 


U 


JJ 


U 


fll 






0 




CJ 


0 




(U 


3 


(U 


0 


JJ 








Vi 


M 




3 




0 


CO 


c; 




0 




Qi 


3 






u 


jj 


3 


rH 




rC 


00 


cn 


c 




3 






•H 


Qi 


?^ 


,Q 


c; 


JJ 




rH 


U 


rH 




JJ 










0 




a; 




JJ 


c 


0 






CO 


1 


Qi 


JJ 


Cu 


JJ 






u 


^3 


0 


CO 




CO 










CO 


c 


a; 


ev 






1-1 

JJ 


Cu 

u 


13 




u 

13 


w 


nc 


er 


0 


li 


00 
C 


JJ 


CO 




>H 


(4H 


ns 


c 


•rl 


ry 






>^ 


?^ 


00 






0 


w 


3 


C 


13 








Cu 


U 


•H 


•H 


>-t 


>-t 




rH 


0 


0 


0 


C 


CO 








ffl 


c 








•H 


a 


CO 


(U 


U 


a; 


3 


X 


CO 




e 


0 


CO 


0 


CO 




•H 




Q 


c 










•H 






JJ 








■U 


(U 


3 


rH 


Qi 


rH 


•H 


rH 


(4-1 


i-H 


JJ 




W 


JJ 


)H 


P 


•rl 












Vi 




(U 




u 


(U 


c 


JJ 


w 


(4-1 




3 


(/] 


Qi 




3 




13 


)H 


CO 


0 


B 


rH 






a; 




0 


rH 




(/3 




0 


m 


0 


W 


M 




13 


CJ 


c 




?>> 


CJ 


13 


•H 


CJ 


•rl 




0 


D* 






c 


0 


JJ 


0 






W 




•H 


u 


3 


3 


•H 




•H 


0 


3 


r-< 


•H 


c; 


> 


JJ 


CJ 




CJ 


?rl 






•H 






0 




. 3 


cn 


•s 


JJ 




rH 


Q* 




u 


JJ 


Q« 




•H 


JJ 


JJ 


•H 


W 


0 


Vi 




O 








?^ 


c 






m 


CO 


U 


>-l 




CJ 




rH 


CO 


}-t 


Vi 






}-t 


JJ 


13 


3; 


Vi 


Qi 


o 


OT 






a; 


CO 


0 


o 




.- 


rH 


•H 


(U 


0 




0 


CO 




CO 


Qi 


> 


CO 


CO 


•rl 


C 


rH 


Vi 


•rl 


3 


•rl 






Q 


E 


e 


en 






CJ 


JJ 


a* 


CJ 


< 


JJ 


u 


w 




U 


0 


e 


PU 


e 


•rl 


u 


CO 


CJ 


OT 


13 






















e 




13 
























ion 


1 




















(/] 






CO 




C 








c 
















JJ 


OT 




















U 






u 




CO 








0 
















CO 


•rl 




















<v 


u 




00 












•H 
















•rl 


C 




















u 


0 




0 




JJ 








JJ 
















U 


•rl 




















•H 


U-l 




M 




C 








CO 
















0 


J 
















en 




U-l 






Pm 




(U 








CJ 
















W 


13 












c 




(U 




U-l 


cn 








e 








3 
















m 


<; 












0 




•H 




O 


(U 




JJ 




CU 








13 




C 












<5 














•H 




U 


(/] 




00 




c 




0 








-. W 




0 










c 




I-H 
















0 


c 


1-* 


(U 








' r-f 












•rl 










0 


-g 


0 












CO 




JJ 


0 


0 


rH 




e 




(U 








(4-1 




JJ 










•H 




0 








w 




u 




CO 


•H 


0 


rH 




Si 




> 








0 




CO 










JJ 












a; 










JJ 




0 




> 




(U 












U 










CO 


CO 


CJ 








r-i 




•X3 




0 


CO 


CJ 


U 




0 




P 








JJ 




3 










•rl 


Qi 


CO 








a 


w. 


a; 


Vi 




M 


CO 






u 












c 




13 


CJ 








CJ 


Vi 


(4H 








B 


(U 




Q} 


CO 


•H 




U-l 




Cu 




rH 








Qi 






00 








0 


01 








nj 


•H 




•H 


rH 


c 


(U 


0 




e 




CO 








e 


Vi 




c; 








Vi 




0 








!< 


u 


a; 


U 




CO 


JJ 






M 




c 








JJ 


r-i 








u 




Vi 














c 




c 


rH 


00 


CO 


c 








0 






r-< 


j-t 


0 


CO 


1-4 




Qi 




< 


rH 


C 








a; 


CO 


(U 


CO 


u 


J^ 


0 




r-) 




•H 






•rl 


CO 


0 


c 


0 


0 








CO 


0 








c 


00 


1-1 


00 


c 


0 


CO 


•H 


C 


0 




JJ 






U 


Cu 


■g 


0 


u 


cc 


u 


RE 


c 


c 


•H 








Q 


CO 




CO 


0 






4J 


0 


0 




CO 






c 


Qi 




•rl 




en 


CO 


0 


0 


4J 






IT) 










•H 




U-l 


CO 


•H 






u 






3 


Q 


CO 


JJ 


c 




Qi 


u 


•rl 


•rl 


CO 






cn 




c 




c 


JJ 


u 


(U 


•rl 


JJ 


u 




.3 




(n 


0 






CO 


)-t 


(4-1 


JJ 


Qi C 


JJ 


JJ 


tH 








0 


0 


0 


0 


CO 




•H 


U 


CO 


CO 




13 






u 


Qi 


u 


U 




0 




^ CO 


^ 


CO 


U 






• 


u 


•H 




•H 


u 


CO 




0 


u 










(U 




JJ 


•rl 


0 






rH 


e . 




o 


0 












w 


JJ 


3 


(U 


u 


cn 


3 


cn 








JJ 




CO 


r-l 


> 


JJ 


a 


0 




3 


3 


OT 










CO 


c 


CO 


•X3 


en 




w 


13 


(U 




U 




c 


13 


JJ 


,0 




3 




0 


B 'ri 


13 


13 


« 










u 


0 


u 


(U 


(U 




< 


[x3 


•rl 




0 




Qi 


3 


CO 


3 




0 


>> 




U 


U 


< 










3 


.H 


3 




M 


0 






JJ 




fU 




U 


JJ 


■ 




0 


CO 


JJ 


u 


(4-1 M 


















JJ 


13 


rH 






c 


U 


•rl 






CJ 




Vi 






(4-1 




•rl 


w 


(4-1 O 


f-H 


' C 


Hi 


OT 








a; 


3 


o; 


CO 


(U 


rH 


CO 


0) 


U 




rH 


M 


M 






0 




CO 


Vi 




CO U 


CO 


CO 


CO 


U 




ON 




JJ 


c 


JJ 


1-1 


u 








1-1 


CO 


Qi 


1-4 


0 


00 


)-t 


•rl 


>-t 


u 


JJ CO 


c 


u 


CJ 


0 




I-H 




r-i 


•i-l 


(i; 


0 


CO., 


U 


•rl 


u 


JJ 




CJ 


Qi 




o 


>-< 


CO 


Qi 


00 


Qi 


•rl 


W 3 


0 


iH 


•rl 


JJ 








CO 


JJ 


JJ 


•H 


> 


c 


U 


CO 


CO 




c 


Vi 


O 


0 




u 


JJ 


>-l 


> 


i-H 




iH 


U 


U 


CO 








U 


m 


CO 


00 


1-1 


3 


(U 


(U 


di 




3 


Qi 


CO 


■g 


3 


•rl 


c 


0 


•rl 




< CO , 


JJ 


Qi 


Qi 


u 






0 


c 


JJ 


(U 


u 


0 


J 


H 


u 




0 


PS 


(U 




(U 




c; 


1 


C 


3 


w > 


CO 


1 


1 


JJ 




u 










1 




u 
1 






o 

1 




u 
1 


H 
1 


CO 

1 


2: 
1 


u 
1 


D 
f 


PU 
1 


CO w 
1 f 


23 

1 










[ Cla 




.J- 

1 


t 


CO 

1 


a* 
1 


t 








1 


1 


1 














































Vi 








ss 






and 




Qi 






































Qi 








0 










03 






















en 
















•rl 








u 






CO 


OJ 


cn 






















3 
















U 








a 








•H 


CO 


cn 








(4-1 


Vi 








(4-1 


0 
















c 




(4H 


cn 


CO 






3 


M 




(U 








0 


3 








0 


(U 














rH 


Qi 




0 


rH 










0 


o 


1-1 










0 










c 






CO 








CO 


00 






CO 


u 








Ot 




u 








en 


<u 


en 






Vi 


(U 


en 




P 








3 


CO 




m 


3 


0 


OT 






(1) 


u 


c 








M 


c 


<u 






U 


00 


oi 




13 








13 








13 




QJ 






JJ 


•H 


(U 








<U 


(U 


•H 






Qi 


0 






•r-l 


•rl 






•rl 






c; 


•rl 


c 


lH 




(U 


CTJ 


M 


00 








• JJ 


00 


u 






JJ 


u 


u 




> 


U 






> 






JJ 


> 


•rl 


a 




CJ 


U 


(U 


CO 








Vi 


0 


c 






Vi 


Qi 


c 




•rl 


c 






•rl 






• w 


•rl 




c 




u 


C 










3 


6 


(U 






3 


JJ 


(U 




13 


c; 






13 


4J 




3 


13. 


4J 


o 




3 




(U 










i-< 


0 


00 






r-l 


Qi 


00 




. C 


00 






■c 


•rl 




i-H 


C 


•rl 


.00 




0 






0 








U 




CO 






U 




CO 




H 


CO 






M 






u 


iH 


3 


CO 




CO 



o 

ERIC 



117 



Amendment (e.g.^ to trace out the impact on KPU of not having certain 
types of longitudinal data available within reasonable cost)^ it should 
not be made prerequisite to the policy impact assessment study recom- 
mended above. 

(We note that an enumeration of the principal agents^ resources^ 
and activities that comprise the entire KPU infrastructure in education 
is a sorely missed feature of the current Databook. The cost of a survey 
to make this type of data available^ however^ together with our conclusions 
regarding the necessity to generate conceptual categories around existing 
policy analysis data, requirements^ suggest that unless NIE wishes to under- 
take two such surveys^ a survey of agents^ resources^ and activities in 
KPU be delayed until the principal configurations have been identified 
and described. ) 

Conclusion 

In this project we explored some of the design requirements necessary 
to create a practicable program for monitoring the KPU infrastructure. 
Due to the scope of our study^ we did not consider the design requirements 
for the KPU system itself although this clearly needs to be considered as 
well. As Sam D. Sieber recently observed: 

A national R&D system^ we are told^ is something that NIE is 
mandated by Congress to nourish and bring to fruition. some- 
thing which nearly all of us endorse--and yet^ no one seems 
inclined to define this system. Clearly it is not something 
that exists in the natural order of things, but something 
that we would like to approximate more closely than at pres- 
ent. These considerations have led me to ask: What might be 
the design requirements of an R&D system in education?. 
Note that I am addressing myself to systemic requirements and 
not to the ultimate intended outcomes of the system, . such as 
equity^ preparation for work, enlightened citizenship, etc. 
To a large extent system requirements can be spelled put inde- 
pei^dently of the desired outcomes.... If present funding re- 
strictions 'continue^ however^ so that these systemic needs 

• 118 , ■ . ■■ 



140 



cannot be met^ then I would urge that we disabuse ourselves of 
the conceit that we are creating a ^'national educational R6eD 
system/' 

National Institute of Education (1975), p. 85 

Both because of the problems and issues discussed in this report and 
because of observations such as it quoted above, we conclude that it is 
not feasible to implement a i),rogram for monitoring the KPU infrastructure 
in education that would be based on a thoroughgoing ^'social indicators/ 
system dynamics" conceptual orientation (or any other type of unitary 
orientation). Instead, the conceptual underpinnings as well as the con- 
crete specifics of the monitoring program must be developed on a step by 
step basis through which the Institute and other KPU agents "learn to moni 
tor and monitor to learn." 
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V A BIBLIOGRAPHY FOR THE INVESTIGATION OF KNOWLEDGE 
PRODUCTION AND UTILIZATION IN EDUCATION 



This bibliography lists those references we found most useful in our 
exploration of the normative structure of knowledge production and utili- 
zation. We present it as a working tool to be used along with the analytic 
framework to assist in NIE's design and implementation of a monitoring 
program. In decisions concerning the selection of entries, we emphasized 
quality rather than quantity. 

The bibliography is divided into six sections: 

• Educational Governance, 

• Systems Literature, 

• Other Methods for Studying KPU, 

• Policy Information Sources, 

• Guides to the Use of Policy Information Sources, 

• Legal Resources and Policy Archives: A Quick Reference 
Guide for Policy Analysts. 

In the first five sections we have annotated those references we 
found most useful and have included simple bibliographic listings of other 
relevant readings. The final section provides a quick reference guide de- 
veloped through the work on this project to legal resources and policy 
archives. 



Educational Governance 

This section covers materials on the structure and operation of agen- 
cies that make up the education system, on the decision-making or affected 
agents, and on the political and governmental environments in which they 
operate. 



"Alternative Futures and Educational Policy," report prepared by the Edu- 
cational Policy Research Center, SRI Memorandum Report EPRC 6747-6, 
Contract OEC- 1-7-0701013-4274, Stanford Research Institute, Menlo 
Park, California (February 1970). 

Becker, J. M. , "The Climate for Change: Factors That Foster Adaptability 
Within the Schools," The North Central Association Quarterly , Vol. 18 
No. 2, pp. 281-286 (Fall 1973). 

Bailey, S. K., Education Interest Groups in the Nation' s Capital (American 
Council on Education, Washington, D.C., 1975). 

Bailey, S. K., "Significance of the Federal Investment in Education R&D," 
Journal of Research and Development in Education , p. 31 (Summer 1969) 

Brickell^ H. M. , Data for Decisions, A Report to NIE (Policy Studies in 
Education, New York, New York, 1974). 

Campbell, R. F., and T. L. Mazzoni, Jr., State Governance Models for the 
Public Schools (The Education Governance Project, Ohio State Univer- 
sity, Columbus, Ohio, 1974). 



Centre for Educational Research and Innovation, Alternative Educational 
Futures in the United States and in Europe; Methods/ Issues and 
Policy Relevance (Organization for Economic Cooperation and Develop- 
ment, Paris, 1972). 

Citizen* s Conference , on State Legislatures, State Legislatures: An Evalu ^ 
ation of Their Effectiveness (Praeger Publishers, New York, New York, 
1971). 

Cohen, M. D., and J. G. March, Leadership and Ambiguity , prepared for the 
Carnegie Commission on Higher Education (McGraw-Hill, Inc., New York, 
New York, 1974). 
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Cohen, M. D. , J. MarclT7 and J, Qlsen, "A Garbage Can Model of Organiza- 
tional Choice," Administrative Science Quarterly , Vol. 17, pp. 1-25 
(March 1972). 

Dewey, J,, Experience and Education (Collier Books, New York, New York, 
1973). 

Downs, A., Inside B^ureaucracy (Little, Brown & Company, Boston, Massachu- 
setts, 1967). 

Eidenberg, E., and R, Morey, An Act of Congress; The Legislative Process 
and the Making of Educational Policy (W. W. Norton & Co., Inc., New 
York, New York, 1969). 

Federal Role in Education , 2nd ed. (Congressional Quarterly Service, Wash- 
ington, D.C., 1967). 

Fulbright, E. R, , and E. C. Bolmeir, Courts and the Curriculum (W. H. Ander 
son Co., Cincinnati, Ohio, 1964). 

Fullan, M., "Overview of the Innovative Process and the User," Interchange , 
Vol. 3, No. 2-3 (1972). 

Fuller, E., and J. B. Pearson, eds.. Education in the States: Nationwide 
Development Since 1900 (National Education Association of the United 
States, Washington, D.C., 1969). 

Gartner, A., C. Greer, and F. Riessman, eds.. After Deschooling, What? 
(Harper & Row, Publishers, New York, New York, 1973). 

Gray, V., "Innovation in the States: A Diffusion Study," American Politi - 
cal Science Review , Vol. 67, pp. 1175-1185 (1973). 

Holzner, B., The Impact of the Federal Research and Development Center Pro - 
gram on American Education (University of Pittsburgh, Pittsburgh, 
Pennsylvania, 1974). 

Jacob, H., and K. Vines, Politics in the American States: A Comparative 
Analysis, 2nd ed. (Little, Brown & Co, , Boston, Massachusetts, 1974), 

Kirp, D, L., and M. G. Yudof, Educational Policy and the Law, Cases and ^ 
Materials (McCutchan Publishing Corp., Berkeley, Calif ornia, 1974). 
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Kirst, M. , ed.. The Politics of Education at the Local, State, and Federal 
Level (McCutchan Publishing Corp.^ Berkeley, California, 1970). This 
book is a collection of articles by educational researchers. It ad- 
dresses the questions of who has influence in school policymaking and 
who is subject to it, how this influence works in the education con- 
text, what the terms are upon which influence is expended, how action 
is concerted by influence, and who aggregates enough '^pieces of in- 
fluence*' so that the total is sufficient to adopt the proposal. An 
article of particular interest is "State Politics of Education" by 
Laurence lannaccone. This article conceptualizes systemic causes of 
a lack of "good feelings" between agents in the academic and research 
communities and agents functioning within the bureaucracy. 

Koerner, J. D» , Who Conrols American Education? A Guide for Laymen (Beacon 
Press, Boston, Massachusetts, 1968). 

Levin, B. , and M. A. Cohen, Levels of State and Related to State Restric - 
tions on Local School District Decision Making (Urban Institute, Wash- 
ington, D.C., 1973). 

Lindblom, C. E., The Policy-Making Process (Prentice-Hall, Inc., Englewood 
Cliffs, New Jersey, 1968). 

Machlup, F., Knowledge Production and Utilization in the U.S. (Princeton 
University Press, Princeton, New Jersey, 1962). 

Marien, M. , and W. L. Ziegler, eds.. The Potential of Educational Futures 
(Charles A. Jones Publishing Co., Worthington, Ohio, 1972). 

Merrow, J., "The Politics of Federal Educational Policy: The Case of Edu- 
cational Renewal," Teachers College Record , Vol. 76, No. 1, pp. 19-38 
(September 1974). Merrow presents an excellent review of USOE- 
congressional struggles over "educational renewal" programs, a good 
example of the lack of cohesion and coordination in the political 
decision-making environment relevant to federal educational policy- 
making. This paper provides important insights into the political 
history of teacher centers. 

National Commission on the Reform of Secondary Education, The Reform of 

Secondary Education: A Report to the Public and the Profession , es- 
tablished by the Charles F. Kettering Foundation (McGraw-Hill Book 
Co., New York, New York, 1973). 

National Institute of Education, R&D Funding Policies of the National Insti - 
tute of Education: Review and Recommendations (Government Printing 
Office, Washington, D.C., 1975). 
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Pearson, J., and E. Fuller, Education in the States: Historical Develop - 
ments and Outlooks (National Education Association, Washington, D.C., 
1969). 

Pedersen, K. G., "State Public School Systems,'* The Book of the States, 
1974-1975 , Vol. XX (Education), pp. 301-312 (The Council of State 
Governments, Lexington, Kentucky, 1974). 

Pincus, J., "Incentives for Innovation in the Public Schools," The Rand 
Corporation, Santa Monica, California (January 1973). 

Price, D. , Who Makes the Lav?s? Creativity and Pov?er in the Senate Com - 
mittees (Shenkman Publishing Co., Cambridge, Massachusetts, 1972). 

Public Administration Service, "Grants Management: State Education Agen- 
cies and the Office of Education," Chicago, Illinois (1971). This 
report is an excellent description of sources of conflict between 
state and federal ecjucation agencies over grants. 

Sarason, S. B., The Culture of the School and the Problem of Change . (Allyn 
& Bacon, ^ Inc. , Boston, Massachusetts, 1971), 

Sieber, S. D., and P. F. Lazarsfeld, "The Organization of Educational Re- 
search in the United States," Cooperative Research Project No. 1974, 
Bureau of Applied Social Research, Columbia University, New York, 
New York (1966). 

Suiranerfield, H. L. , Power and Process, The Formulation and Limits of Fed - 
eral Educational Policy (McCutchan Publishing Corp., Berkeley, Cali- 
fornia, 1974)-. 

Task Force on Resources Planning and Analysis, "Building Capacity for Re- . 
newal and Reform: An Initial Report on Knowledge Production and Ut- 
ilization in Education" (National Institute of Education, Office of 
Research and Development Resources, Washington, D. C, 1973). Dis- 
cusses the need for an investigation and revision of concepts con- 
cerning research and its use in the educational system. Suggests 
that the paradigm of R&D is too narrow and that more appropriate 
would be a revised concept that would include how and by whom prob- 
lems are formulated, a range of likely resources for solving them, 
and the organizational life of operating systems that will affect 
the possibility of, implanting a solution. This paper is the con- 
ceptual basis on which NIE initiated a series of studies for fui^ther ' 
investigation of the knowledge production and utilization system in 
education. 
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U.S. Office of Education, Educational Research and Development in the 
United States (Washington, D. C. , 1969). 



U.S. Office of Education, Reinforcing the Role of the States in Education , 
The Second Annual Report of the Advisory Council on State Departments 
of Education (Government Printing Office, Washington, D.C. 1967). 

Wirt, F. M., Contemporary School Turbulence and Administrative Authority , 
paper presented at the David W. Minar Memorial Conference on Problems 
in the Politics and Governance of the Learning Community, Northwestern 
University, 30 October-1 November 1974. 

Wirt, F. M. , and M. W. Kirst, The Political Web of American Schools (Little, 
Brown & Co., Boston, Massachusetts, 1972). Wirt and Kirst provide a 
comprehensive analysis and description of the political aspects of 
American education. The authors use a nontechnical systems framework 
encompassing interactions of schools, governments, and the community, 
as well as comparative, aggregate data and case studies. A survey of 
recent work by educational scholars and political scientists on the 
. subject is also provided. 

Zeigler, L. H. , and M. K. Jennings, Governing American Schools: Political 
Interaction in Local School Districts (Duxbury Press, Belmont, Cali- 
fornia, 1974). 

Zeigler, L. H. , and K. F. Johnson, "Educational Innovation and Politico- 
Economic Systems," Education and Urban Society , Vol, 1, No. 2 (Feb- 
ruary 1969). 

Zeigler, L. H. and- K. F. Johnson, The Politics of Education in the States 
(Bobbs-Merrill Co., Inc., Indianapolis, Indiana, 1972) . Ziegler and 
Johnson provide an analysis of state educational systems and political 
variables as they affect educational decisions at the state level. 
This is a study largely based on statistics. 
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Systems Literature 

In this section we cite the literature that undergirds our efforts to 
build a systems-oriented analytical framework for understanding KPU in 
education, and that deals with other attempts to use the systems orienta- 
tion in understanding complex social systems. 



Ackoff, R. L,, and F. E. Emery, On Purposeful Systems (Aldine-Atherton, 
Inc., New York, New York, 1972). 

Ashby, W. R., An Introduction to Cybernetics (Chapman & Hall, Ltd., London, 
England, 1956), 

Beckner, R. , "I Don't Know PPB at All," Policy Sciences , Vol, 2, pp. 301- 
304 (1971). 

Beer, S., Platform for Change (John Wiley & Sons, New York, New York, 1975). 

Berelson, B., and G. A. Steiner, Human Behavior (Harcourt, Brace & World, 
Inc., New York, New York, 1964). 

Blau, P. M. , and W. R. Scott, Formal Organizations (Chandler Publishing 
Co., Scranton, Pennsylvania, 1962). Blau and Scott present a good 
literature review, historical perspective, and conceptual framework 
for the thinking and research done by organization theorists prior 
to 1962. Chapter 2, Part 2, "Typologies of Formal Organizations," 
was quite useful in the development of definitions for our project. 

Boulding, K. , "General Systems Theory — The Skeleton of Science," Management 
Science , Vol. 2, pp. 197-208 (1956). Boulding' s article, one of the 
classic discussions of hierarchies in general systems, is useful for 
conceptualizing levels of importance, issues of complexity, and the 
limits of current systems methodologies. 

Ik 

Brickell, H. M. , and S. Wong, Conference Report; Dissemination of NIE 

Sponsored Products (Henry Chauncey Conference Center^ Princeton, New 
Jersey, 6-7 September 1973). Brickell and Wong point out the dif- 
ficulties inherent in attempting to apply systems planning to the 
organization of reality in the field. They contrast the performance 
roles that the government planned for the laboratories, centers, and 
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publishers with the actual functions these agents found necessary. 
The finding that these agents were not to be neatly fitted into the 
simple categories of merely researching, developing and evaluating 
or distributing led Brickell and Wong to conclude that the RDDA 
division is not a useful dimension along which to divide the three 
KPU agents investigated, since each performs all of these functions 
in one way or another. 

Bush, v., "As We May Think," Atlantic Monthly , Vol. 176, No. 1 (July 1945). 

Churchill, S., "Modelling a National Education R&D System: A Conceptual 

Framework," prepared for National Institute of Education, Washington, 
D.C., by the Ontario Institute for Studies in Education, Ontario, 
Canada (draft report, 16 January 1974). This unpublished consultant 
paper, prepared for NIE, was part of the background for this research. 
One of its significant contributions is a discussion of the need for 
coordination within the "infrastructure" of KPU in education. 

Churchman, C. W. , The Systems Approach (Delacorte Press, New York, New 
York, 1968). 

Fischer, D. H., Historians' Fallacies: Toward a Logic of Historical 

Thought (Harper & Row Publishers, New York, New York, 1970). 
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Review (1971). 
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in Educational Administration (Rand-McNally & Co., Chicago, Illinois, 
1969). 

Cuba, E. C, and D. L. Clark, The Conf igurational Perspective; A Challenge 
to the Systems View of Educational Knowledge Production and Utiliza- 
tion (School of Education, University of Indiana, Bloomington, 
Indiana, 1974). Cuba and Clark discuss the origins and effects of 
... "systems" thinking on KPU policy and argue f or the desTrability of 
abandoning the "systems view." They suggest that the KPU system is 
composed of conf igurations , of actors and must be investigated as such, 
not as the composite of members of a priori categories* 
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Hage, J., and R. Dewar, "Elite Values Versus Organizational Structure in 
Predicting Innovation," Administrative Science Quarterly , Vol. 18 
(1973). 



Hall, A. D., A Methodology for Systems Engineering (Van Nostrand Reinhold 
Co., New York, New York, 1962). Hall's work is one of the standard 
references on the systems method of analysis. This book provides 
several chapters that are useful for design of studies of social sys- 
tems. It also provides comprehensive examples of various quantita- 
tive tools. 

Hoos, I. R. , "Can Systems Analysis Solve Social Problems?" Datamation , 
pp. 82-92 (Jvne 1974). 

Immegart, G. L., and F. J. Pilecki, An Introduction to Systems for the 
Educational Administrator (Addison-Wesley Puuiishiiig Co., Reading, 
Massachusetts, 1973). This brief book provides an excellent back- 
ground on both systems theory and some of its applications to educa-* 
tional administration. Analytical approaches suggested in the con- 
ceptual portions of the book are often parallel to this stuiy, and 
complement our hypothesis that systems theory concepts can yield 
valuable tools for dealing with and investigating governance issues. 

Jensen, G. E., Problems and Principles of Human Organization in Educational 
Systems (Ann Arbor Publishers, Ann Arbor, Michigan, 1969). 

Las swell, H. , "From Fragmentation to Configuration," Policy Sciences , Vol. 
2, pp. 439-446 (1971). 
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Cross-Professional and Cross-Cultural Communication , Department of 
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Miller, J. G. , "Living Systems: Basic Concepts/' Behavioral Science , Vol. 
10, No. 3, pp. 193-237 (July 1965); and "Living Systems: Structure 
and Process," Behavioral Science, Vol. 10, No. 4, pp. 337-411 (October 
1965). The Miller papers present a comprehensive discussion of that 
part of General Systems Theory dealing with "living systems," of 
which EKPU as a social system is a part. The author uses clear and 
nonmathematical definitions of basic systems concepts and rules, and 
takes an organismic perspective when describing systems. The dis- 
cussion of structure and process in "living systems," as a part of 
General Systems Theory, is followed by 165 "cross-level hypotheses" 
that organize multidisciplinary research findings into systems con- 
cepts. These articles provide useful background for researchers and 
policymakers, with numerous implications for the further study of 
process and regulation in EKPU as well as other areas. 

National Center for Educational Statistics, The State Education Agency, A 
Handbook for Standard Terminology and a Guide for Recording and Re - 
porting Information About State Education Agencies (Washington, D.C., 
1971). 

Patee, H. H. , ed.. Hierarchy Theory (George Braziller, Inc., New York, New 
York, 1973). Patee has collected essays on complex systems and hier- 
archy theory from disciplines such as biology, physics, and social 
organization. Although these essays are separate, self-contained 
monographs directed specifically at the interests of their respective 
fields, Patee draws on this information through the. use of a post- 
script to discuss the themes and general research problems in the 
investigation of hierarchy theory that emerge from across the col- 
lected essaySo 

Porter, A, Cybernetics Simplified (Barnes & Nobel Books, New York, New 
York, 1969). ^ 

Rivlin, A., Systematic Thinking for Social Action , the 1970 H, Rowan 
Gaither Lectures at the University of California, Berkeley (The 
Brookings Institute, Washington, B.C., 1971). 

Sneath, P.H.A., and R. R. Sokal, Numerical Taxonomy (W, H. Freeman & Co., 
San Francisco, California, 1973). 

Sokal, R. R., "Classification: Purposes, Principles^ Progress, Prospects/ 
Science , Vol. 185, No. 4157, pp. 1II5-1123 (27 September 1974). 

Tykociner, J. T., Descriptive Inventory of the Arts and Sciences , Depart- 
ment of Electrical Engineering (University of Illinois, Uxbana, 
Illinois, 1967). 
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U.S. Office of Education, National Center for Educational Research and 
Development Taxonomy (Washington, D, 1969). 

Van Gigch, J. P,, Applied General Systems Theory (Harper & Row Publishers, 
New York, New York, 1974). This book is of interest primarily as an 
introductory text requiring of the reader little or no expertise in 
higher mathematics. 

Von Foerster, H., et al.. Cybernetics of Cybernetics , Biological Computer 
Laboratory (University of Illinois, Urbana, Illinois, 1974). 



other Methods for Studying KP U 

This section provides readings on approaches to the inquiry into KPU 
that differ from the particular analytic approach around which our project 
was initially oriented. These include studies using R&D, RDDA, and 
problem- centered models of educational research and development. 



Allison, G, T, , Essence of Decisions-Explaining the Cuban Missile Crisis 

(Little, Brown & Co,, Boston, Massachusetts, 1971), Although it uses 
an international policy crisis as a case study, this book has become 
something of a classic for students of educational polic3miaking. It 
contrasts three models of decision making: rational actor, organiza- 
tional process, and governmental politics--showing the strengths, 
weaknesses, and benefits of each, 

Anderson, R, C, and D, Ausubel, eds,. Readings in the Psychology of Cogni - 
tion (Holt Rinehart and Winston, Inc,, New York, New York, 1965). 

Birdwhistell, R, L,, Kinesics and Context: Essays on Body Motion and Com - 
munication (University of Pennsylvania Press., Philadelphia, Pennsyl- 
vania, 1970), 

Coser, L , , The Functions of Social Conflict (The Free Press, New York, New 
York, 1956), . 

Ewald, W, R,, Jr., "Graphics for Regional Policy Making^" a preliminary 
study for the National Science Foundation, Washington, D,C, (August 
1973), Ewald discusses requirements and information technologies for 
policymaking decision aids. He provides surveys of some existing 
systems. The ACCESS in Santa Barbara, Calif ornia, receives particular 
emphasis, . ► . 

Fischer, D, H, , Historians' Fallacies; Toward a Logic of Historical Thought 
(Harper & Row, Publishers, New York, New York, 1970) . 

Goffman, E,, Presentation of Self in Everyday Life (Doubleday & Co., Inc, 
New York, New York, 1959), 



132 



153 



Havelock, R. , The Change Agent's Guide to Innovation in Education (Educa- 
tional Technology Publications, Englewood Cliffs, New Jersey, 1973). 
Havelock develops a six- step approach to the adoption of a linear, 
rational model of creating change in the educational delivery sys- 
tem. 

Havelock, R. G. , Planning for Innovation Through Dissemination ' and Utiliza - 
tion of Knowledge (University of Michigan, Ann Arbor, Michigan, . 1971). 

Hempel, C, G., Fundamentals of Concept Formation in Empirical Science , 

Foundations of the Unity of Science Series, Vol. 2, No. 7 (University 
of Chicago Press, Chicago, Illinois, 1952), 

Kap Ian, A. , The Conduct of Inquiry; Methodology for Behavioral Science 

(Chandler Publishing Co., Scranton, Pennsylvania, 1964), Kaplan prob- 
ably provides the single most important reference for designing a 
monitoring program for KPU. He systematically points out the dis- 
tinctions between laws and policies and their interpretation in con- 
text, between logical descriptions of reality and the logic that 
reality is actually following, and between theories and empirical 
generalizations. The book also provides an excellent explanation of 
the role of values and goals in the actions central to monitoring any 
field. 

Lindblom, C. E., ''The Science of Muddling Through," Public Administration 
Review, Vol. 19, pp. 79-88 (Spring 1959), Lindblom argues that' the 
operational realities in which policymakers act do not permit much 
use of rational-analytical models of policy development. 

Mu rphy , J , T . , State Education Agencies and Discretionary Funds Grease the 
Squeaky Wheel (Lexington Books, Lexington, Massachusetts, 1974). This 
book is a study of what happened when the federal government sought to 
strengthen the education agencies of a' number of states. In his in- 
tensive analysis of Massachusetts, New York, and South Carolina, 
Jerome Murphy finds that the effort did little to stimulate the pur- 
poses of the original act, although under some circumstances the 
states used the law to facilitate their own goals. By and large, he 
states, the federal effort results in states' doing more of the. same, 
a "same" which reflected in each state differing combinations of 
power and political contributions: first in its discussion of how 
agencies' behavior stemmed not from ad hoc factors, but from general 
causes common to all organizations. The work is therefore as much a 
study in organizational theory as in political science. Secondly, 
the author provides recommendations for the utility of revenue- sharing 

prog rams^ as we ll as__sJbo.wlngu-what--can^be-expected— i-f— th — — ~ 
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government makes revenue- sharing an important part of its policies 
of education (Annotation is quoted from the catalog of Lexington 
Books). 

Nagel^ E.^ Structure of Science, Problems in the Logic of Scientific Ex - 
planation (Hatcourt Brace Jovanovich, Inc., New York, New York, 1961), 

National Advisory Council on Education Professions Development, "Search 

for Success--Toward Policy on Educational Evaluation," Report to the 
President and the Congress of the United States, Washington, D.C., 
June 1974, 

Ozbekhan, H,, "The Emerging Methodology of Planning," Fields Within Fields , 
No. 10 (Winter 1973-74), This article clearly describes the currently 
evolving "rational planning model, " 

Popper, K, R,, Conjectures and Refutations: The Growth of Scientific Knowl - 
edge (Harper & Row Publishers, New York, New York, 1963). 

Schmidtlein, F. A., "Decision Process Paradigms in Education," Educational 
Researcher, Vol, 3, No. 5, pp, 4-11 (May 1974). 

,Sen, A, K. , Collective Choice and Social Welfare (Holden-Day, Inc., San 
Francisco, California, 1970), 

Simon, H, A,, The Sciences of the Artificial (MIT Press, Cambridge, Massa- 
chusetts, 1969), 

Vickers, Sir Geoffrey, Science and the Regulation of Society , Occasional 

Papers (The Institute for the Study of Science in Human Affairs, Col- 
umbia University, New York, New York, 1970). 

Wirt, F. M., et al,. Introductory Problems in Political Research (Prentice- 
Hall, Inc, Englewood Cliffs, New Jersey, 1970). 

Wirt, J., A, Lieberman, and R, Levien, "National Institute of Education: 

Methods for Managing Practice-Oriented Research and- Development, " The 
Rand Corporation, Santa Monica, California (1971), This report was . 
generated to provide HEW with information on alternative management 
strategies that might be used in the support of educational R&D by the 
then-proposed National Institute of Education, The authors describe, 
the application of eight paradigms of management as used in three 
federal agencies for the management of their practice-oriented re- 
search and development. 
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Policy Information Sources 

This section provides references to sources of information found 
useful in our applications of the analytic framework. It includes the 
following four general categories: 1. archives of formal policy that 
are available; 2. various interpretations of policy^ such as statistics 
and formal opinions; 3. specialized policy reporting news services; and 
4. directories and abstracting services tha.t cover information relevant 
to KPU. 



1972 Census of Governments, Public Employment: Employment of Major Local 
Governments, Bureau of the Census , Vol. 3, No« 1 (Government Printing 
Office, Washington, D. C. ). 

1972 Census of Governments, Government Finances: Finances of School Dis - 
tricts^ Bureau of the Census , Vol. 4, No. 1 (Government Printing Of- 
fice, Washington, D. C). 

Code of Federal Regulations (Government Printing Office, Washington, D. C). 
The Code of Federal Regulations (C.F.R. ) is the codification of the 
regulations emanating from the federal administrative agencies as 
based' on their statutory authority.. It is divided into 50 titles 
and is updated annually. 

Congressional Information Service, CIS/INDEX (Government Printing Office, 
Washington, D.C, 1970). This service provides references and ab- 
stracts to every important document printed by the federal government. 
It is updated monthly and may be accessed both manually and by com- 
puter. 

Congressional Quarterly Service (Congressional Quarterly, Inc., Washington, 
D.C., 1945- ). The Congressional Quarterly publications are a good 
source for an overview of congressional activity and the primary 
chronicle of information on lobby activity at the federal level. 
This service provides three publications: The Weekly Report , the 
Quarterly Index , and the Almanac . The Weekly Report is a magazine 
that focuses on congressional and political activity. It reports 
the full content of presidential press conferences, major statements, 
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messages, and speeches. The "Congressional Box Score," a useful ta- 
ble published in each issue, gives a running report of the status of 
the major proposed legislation in Congress. The Quarterly Index and 
the Almanac are both drawn from the information in the Weekly Report . 
As the title implies, the Quarterly Index is simply an indexing of 
the Weekly Report . The Almanac is a reorganization and summation of 
year's issues and activities in the previous Congress as reported in 
the Weekly Report . This publication is organized by topics and pro^ 
videsan excellent capsulization of the activities and interests of 
the U.S. v'^ongresses. 

Corpus Juris Secundum (C.J.S. ) (West Publishing Co. , St. Paul, Minnesota). 
American Jurisprudence (Am. Jur. ) (Lawyers Cooperative Publishing Co. 
Rochester, New York). C.J.S. and Am. Jur. are the two encyclopedic 
references to American law. C.J.S. is a complete restatement of the 
law, which cites in footnotes all cases that support a particular 
point of law from 1658 to date. Am. Jur. offers a comprehensive re- 
statement of the law; however, it cites only selective decisions in 
its footnotes. Both encyclopedias may be found in the normal law 
library. 

Current Index to Journals in Education , National Institute of Education 
(The Macmillan Co., New York, New York, 1969- ). 

Current Index to Research in Education , National Institute of Education 
(Government Printing Office, Washington, D.C., 1966-). The Current 
Index to Research in Education (R.I.E. ) and the Current Index to 
Journals in Education (C.I.J.E.) are the components of the ERIC Sys- 
tem, an abstracting service for publications concerning the field of 
education. The R.I.E. covers speeches, papers, and reports not pub- 
lished in the journals. C.I.J.E. covers the entries related to edu- 
cation in over 700 periodicals. These files are updated monthly and 
may be searched both manually and by computer. 

ERIC Clearing House on Information Resources, "A Guide to Educational Re- 
sources, 1975-76," Stanford Center for Research and Development in 
Teaching, School of Education, Stanford University, Stanford, Cali- 
fornia (Fall 1975). This 33-page guide is a concise, up-to-date di- 
rectory of selected sources of education information. 

Harris, S. P., State Departments of Education, State Boards of Education , 
and Chief State School Officers (Office of Education, Department of 
Health, Education and Welfare, Washington, D. C, 1973). 

Lawyer's Committee for Civil Rights Under Law, A Study of State Legal 
Standards for the Provision of Public Education , prepared for the 
National Institute of Education (1974).* 
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National Educational Association, Ranking of the States 1973 (National 
Educational Association, Washington, D.C, 1973), 

Office of Education, Digest of Educational Statistics (Government Printing 
Office, Washington, D.C., 1962- ). 

Office of Education, The State of State Departments of Education , The 

Fourth Annual Report of the Advisory Council on State Departments of 
Education (Government Printing Office, Washington, D.C, 1969), 

Paisley, W, , et al,. The Status of Educational Research and Development 

in the United States: 1975 DATABQOK (NIE, Washington, D.C; prepub- 
lication version. May 1975), 

Ratliff, S,, ed. , State Education Journal Index (Fort Collins, Colorado, 
1963- ), 

Standard Education Almanac (Academic Media, Los Angeles^ California, 1968- ). 

State Codes. The state code provides the text of state statutory law, 
which in policy analysis entails the state- level governance. The 
code, especially the annotated version, is often a good starting 
point for information gathering. It will either state the full set 
of state- level policies, agents, resources, and activities that are 
needed to begin policy analysis, or it will direct the researcher to 
the appropriate sources. 

United States Code Annotated (West Publishing Co., St. Paul, Minnesota), 
This commercial publication reproduces the official United States 
Code , but annotates the statutes with digests of court decisions, 
citations to relevant administrative regulations, and a brief out- 
line of each provision's legislative history. 

United States Code, Congressional and Administrative News (West Publishing 
Co., St. Paul, Minnesota, 1941-). This publication collects the more 
important federal legislative history materials and binds them with 
the text of the enactment. Excerpts from hearings or committee re- 
ports of federal laws enacted since 1941 will often be found in this 
service. 

U.S. Library of Congress, Monthly Checklist of State Publications , 56 Vols. 
(Government Printing Office, Washington, D.C., 1910-). 

Wanger, J., Directory of Educational Information Resources (CCM Information 
Corporation, New York, New York, 1971). 
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Winchell^ M.^ Guide to Reference Books , 8th ed, (American Library As- 
sociation^ Chicago^ Illinois^ 1967). This book is one of the most 
exhaustive collections of reference works. Completeness and organi 
zation make this the handbook of reference librarians. 



138 



Guides to the Use of Policy Information Sources 



In this section we cite a variety of handbooks and guides in legal 
and reference research that explain techniques needed to use more fully 
the types of materials described in Section D of this bibliography. 



Cohen, M., Legal Research in a Nutshell (West Publishing Co._, St. Paul, 
Minnesota, 1968). 

Doyle, J., '^Searching Education Literature, A Brief . Guide, " Reference 
Quarterly , Vol. 11, No. 3 (Spring 1972). 

Foskett, D. J., How to Find Out:> Educational Research (Pergamon, Oxford, 
and New York, 1965). 

Harvard Law Review Association, A Uniform System of Citation , 11th ed. 
(Harvard Law Review Association, Cambridge, Massachusetts, 1967), 

Jacobs tein, J. M., and R. M. Mersky, Legal Research Illustrated , 4th ed. 
(Foundation Press, Mineola, New York, 1973), This book, a compre- 
hensive, clear guide to legal research, is an abridgment of Funda - 
mentals of Legal Research by Erwin H, Pollack, 

Katz^ W, A., Introduction to Reference Work Volume II: Reference Services 
and Reference Processes , 2nd ed, (McGraw-Hill Book Co,, New York, ' New 
York, 1974). This book is a basic text for serious students of ref- 
erence research. It provides standard reference methods and sources 
to initiate the researcher to logical, comprehensive approaches to 
information gathering. The author provides extensive footnotes and 
lists of suggested readings to complement the material presented. 

Price, M, 0,, and H, Bitner, Effective Legal Research; A Practical Manual 
of Law Books and Their Use , 3rd ed, (Little, Brown & Co., Boston, 
Massachusetts, 1969), Price and Bitner offer a thorough description 
of all the tools of the legal profession. The book is indispensable 
for its in-depth study of reference works and indexes that aid in 
finding U, S, law. Of particular interest are Chapters 5, "Legislative 
Histories" (pp, 56-73) and 6, "Federal Statutes: Index and Tables" 
(pp, 74-92), 
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Rezny, A. A. ^ and K. Remmlein, A Schoolman in the Law Library (Inter- 
state Printers and Publishing, Danville, Illinois, 1962). 



Roalfe, W. , ed., How to Find the Law, with Special Chapters on Legal 

Writing , 6th ed. (West Publishing Co., St. Paul, Minnesota, 1965), 

Rombauer, M. D., Legal Problem Solving, Analysis, Research and Writing , 
2nd ed. (West Publishing Co., St. Paul, Minnesota, 1973). 

Todd, A., Finding Facts Fast; How to Find Out What You Want to Know Im - 
mediately (William Morrow & Co., New York, New York, 1972). 

West Publishing Company, West's Law Finder, A Research Manual for Lawyers 
(West Publishing Co.. St. Paul, Minnesota, 1967). 
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Lefial Resources and Policy Archives; A Quick Reference 
Guide for Policy Analysts 

One skill which policy research in education requires of the analyst 
is basic competency in legal reference work for both the general acquisi- 
tion of policy information and the resolution of particular questions. 
This requires the understanding of definite techniques. Unfortunately, 
however, although these techniques are not difficult to acquire and em- 
ploy, they are possessed by few educational researchers or policy analysts 
since legal reference work is usually taught only in law school. There- 
fore, the attached table is provided to give researchers unfamiliar with 
legal research a quick reference guide for use in the law library. 
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Coiranents 


1 


See the second comment in Row IV of this ta- 
ble concerning federal depository libraries. 

For federal legislative history, the first 
place to look is always in the United States 


Code. Congressional and Administrative News. 
This publication collects the more important 
federal legislative history materials and 
binds them together with the text of the en- 
actment. 


a 

V 

ij 
ui 


B. When only the popular natue of the enact- 
ment is known 

Consult the Table of Popular Nataes (see 
Row V of this table) and get the pub- 
lic law number. 

Follow the instructions for C in this 
Box. 

C. When only the public law number is known 

Consult the Daily Dipest, which is In 
the Congressional Record (and compiled 
annually). 

Locate Information about the legisla- 
tive history by the PL number. 
OR 

Consult the Legisl.attve Cross-Reference 
table (see Row HE, B, of this table) 
and obtain the citation of the statute 
in the code. 

Follow the steps outlined in A of this 
Box. 

D. For bills that did not or have not yet 
become law 

Locate the bill number from the subject 
index of the annual index of the Con- 
gressional Record. 

Still using the Congressional Record An- 


nual Index, consult the History of 
Bills and Resolutions table for ref- 
erences to legislative history docu- 
ments. 

Obtain the documents as outlined tn Row 
IV of this table. 

State 

For each state in question, contact the ref- 
erence desk at the state's official law li- 
brary. 


Federal 

A, Finding the original text or revisions to 
the text of a bill 

Obtain the number of the bill as indi- 
cated in Row III, A, of this table. 

Use that as the official document number 
to obtain the text from any federal de- 
pository library. 

B, Finding the transcripts of the hearings 
Obtain the number of the bill as indi- " 

cated in Row III, A, of this table. 

Consult the U.S. Monthly Catalog of Gov- 
ernment Publications (compiled annu- 
ally), using bill number to determine if 
hearings were held and recorded. 

Obtain documents, if any, from a federal 
depository library. 


Sources 




Federal 

Official federal documents 
Congressional Record 

United States Code, Congressional and Ad- 


• 

■ 

• 


1 Reference Need 1 


VI (Continued) 


VII Sources of Legislative History 

A. Finding the original text 
or revisions to the text 
of a bill 

B. Finding the transcripts of 
the hearings 

C. The committee reports , 

' D. Transcripts of floor debates 
E. binding the original text of 
the enactment 
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Appendix A 



A BIBLIOGRAPHIC ESSAY OF GENERAL SYSTEMS THEORY 
AND KPU IN EDUCATION 

Our analytical framework requires an extremely broad conceptual 
foundation if it is to succeed in describing the complex structure of 
governance in KPU in education. Because general systems theory (GST), 
incomplete as it is, provides a basis for various theories of organiza- 
tional behavior as well as criteria for analysis of systems at many 
different levels and of many different kinds, it is a prime candidate 
for the conceptual role. Cybernetics theory, as one of its most important 
subsets, deals quite fundamentally with issues of control, regulation, 
and '^deciding"; it will play a major part in developing the first part of 
the framework. 

The .body of theory and methodology that makes up systems science 
has grown rapidly since pioneering work was done by von Bertalanffy, 
Weiner, von Neumann, Shannon, and others in the early 1940s. Most of 
its perceived usefulness has been in areas of technology design and 
analysis; in various approaches to models and simulation; and, less 
successfully, in planning and -decision-making models for social and 
institutional systems. Its successes in this latter area have generally 
been limited by the complexity of the topics to which it has been applied. 

In this appendix we present several overviews directed toward major 
applications, particularly in education, and a brief review of the main 
causes for success and failure. 
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While GST remains mostly a synthesis of attempts to demonstrate a 
set of laws for all kinds of systems, its mos^; successful applications 
have been somewhat limited. The major successes have been in the design 
and analysis of nonliving, technological systems-- the artifacts of human 
invention. A. D. Hall (1962) describes in detail this methodology, 
called systems engineering, and suggests extensive applications for 
dealing with complex social systems. (There are now a number of softer 
systems engineering books available that present similar approaches 
without elegant mathematical trappings.) The power of systems engineerin 
is clearly demonstrated by the rapid development and proliferation of 
computers and sophisticated military technology, and by a general trend 
toward the cybernation of mechanistic processes. Computers, in particu- 
lar, have had a profound effect on institutional management and planning, 
as a budgeting aid, and on the nature of KPU research itself in education 
at all levels. 

The general advent of high-speed computation automata has created a 
medium for designing complex mathematical models and simulating their 
behavior, over time and under a variety of conditions, without disturbing 
the actual system that is being modeled. Systems dynamics, a technique 
first used by Forrester (1961) in simulating industrial system processes, 
has proved useful as a methodology for examining change in systems where 
variables are quantifiable and the relationship between elements are 
usually higher-order difference equations; queuing theory and other 
event-dependent models have been applied to problems of human-machine 
interaction and scheduling in social systems (Gordon, 1969). Various 
specialized applications of both major modeling approaches have found 
their way into the education research and administration environment. 



Deductive, as well as inductive, approaches to GST abound in the area's 
literature. 
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Systems analysis as a general planning tool is discussed from a 
number of perspectives by Ozbekhan (1969) and others. One often-used, 
but still controversial technique, the planning-programming- budgeting 
system (PPBS) , is used in both private and public educational organiza- 
tions. Critical path method (CPM or PERT) is often used for analysis 
of scheduling problems, in education research design, and in school 
operations as well as in the noneducation area (Immegart and Pilecki, 
1973, p. 18). Many other techniques for dealing with "systemic" kinds 
of problems--f rom decision theory to gaming to "information theory--are 
used in education, usually emulating their application elsewhere; we see 
no need to list all of these here. 

The primary impact of systems theory in education (both KPU and 
non-KPU) has been in the area of administration and organizational be- 
havior. A. Downs (1967) developed a theory of bureaus and bureaucratic 
behavior; many of the "laws" and hypotheses described in it are relevant 
to the analysis of KPU administrative organizations; these laws are also 
analogous to a group of GST hypotheses, Immegart and Pilecki (1973) 
provide a comprehensive overview of system theory and its relevance for 
education administrators. In an earlier study, Immegart (1969)- develops 
four distinct, but interrelated, analytical frameworks for classifying 
education administrative behavior; these were quite helpful in developing 
our initial framework of the governance system. - Churchill (1974) pro- 
poses and discusses a cybernetics-based KPU analytical framework that is 
also partially congruent with our approach. H. D. Gideonse (1968) dis- 
cusses the staging of administrative events for managing education 
research and development; Schalock and Sell (1972) construct a systems 
framework for the analysis of research, development, diffusion, and 
evaluation in education (RDD6cE-- another label in vogue for KPU). Organiza 
tional behavior theories range along a continuum bounded on one side by 
rational decision-making models (Maxsbn and Sistrunk, 1973 is typical), 
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and on the other by more or less random decision models (Cohen, March and 
Olsen, 1972; Lindblom, 1959). A systems approach to the problem of de- 
scribing how schools behave in a political environment is explored by 
Kirst and Wirt (1972) in an extension of Easton's (1965) work on politi- 
cal science in a systems context. Most, if not all, include some basic 
systems concepts in their analytical frameworks. 

Development of comprehensive futures planning methodologies, based 
primarily in systems analysis, now attempts to account for systemic 
changes in the environment from which education theory, practice, and 
facility requirements evolve. One such study, undertaken by Project 
Simu School, incorporates a systems flow chart describing various impacts 
of societal change on learning theory and curriculum changes in develop- 
ing a facilities planning model (Leu, Ford and Cornish, 1974) . 

Generally, attempts to deal with more complex (i.e., whole- system) 
or more substantive (e.g., curriculum reform) issues have not been as 
well received. The reasons for failure are many, but foremost is the 
incompleteness of various systems approaches. Misuses of systems hypoth- 
eses, misunderstanding and imprecise use of systems language (e.g., PPRs 
in this study), and especially promises of results considerably beyond 
the current state of the art (Hoos, 1974) contribute to dissatisfaction 
with the systems perspective. 

Success is quite high in uses for conceptual theory building that 
often implicitly (or explicitly) uses certain subsets- of existing systems 
theory. This usefulness is marked in preliminary research and planning 
steps: in definition and description of basic system elements and their 
attributes, in general discussion of interaction between a system and its 
environment, in pictorial representation of connections between elements 
(e.g., flows of money in KPU, information channels and networks, and the* 
like) , and in determining procedures for more detailed analysis. Models 
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and maps of the sys tem--occasionally mathematical, but more often 
graphical-- are the typical results from this sort of work. At such high 
levels of aggregation and complexity, however, model validation is often 
difficult and may fall short of expectations for reliability and predicta- 
bility. 

More focused applications of systems methodologies are more success- 
ful, particularly in areas of budgeting, personnel management, and narrowly 
defined technologies (e.g., audiovisual equipment design) because of the 
relative simplicity of these systems, the low number of variables con- 
sidered, and the usually explicit and quantifiable objectives that such 
systems are designed to satisfy. Add to most of these efforts a complex 
of human variables, a soupcon of qualitative goals, or a requirement for 
great variety, and the problems concomitant with more holistic approaches 
reappear. 

An unfortunate barrier to success lies within the general resistance 
toward the technique and the jargon of systems: analysts may provide 
relatively elegant, technically rigorous models for their clients; these 
models are more often than not incomprehensible and useless to policymakers 
who must weigh issues and make decisions in a highly volatile political 
marketplace. There are major communication problems, not unlike those 
between scientists of different specialties, between most major parties 
in systems-based social analysis. 

Systems theory as an analytical method and predictive tool is 
currently limited. Nevertheless, it appears to have great value as a 



tool for insight into the complexity and multilevel problems and per- 
spectives that are part of KPU. 
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Appendix B 



EDUCATIONAL KPU: WHAT KIND OF SYSTEM? 

Before a program can be built for monitoring social change, some 
kind of a model of the system to be monitored is necessary. NIE, in its 
1973 position paper, "Building Capacity for Renewal and Reform," charac- 
terized conceptual problems and goals for such^^a monitoring effort: 

• We have lacked the data base and the understanding of 
systems dynamics needed for effective, rational policy- 
making (p. 65) . 

• This program is intended to establish an internal NIE 
capability to monitor the external R6cD system and the 
operating system in education (p. 67). 

• [A] conference on alternative conceptualizations of the 
knowledge production and utilization system [will be 
held] as a first step towards a better understanding of 
the knowledge production and utilization process (p. 69). 

While the question of what sort of system educational KPU might be is 
clearly open for inquiry, the sense that there is a KPU system is never- 
theless pervasive. Describing the steps to build a monitoring program, 



NIE goes on to say: 

The concept of "monitoring" is borrowed from the literature 
on social indicators. That literature was originally focused 
largely on the identification and measurement of outcomes at 
a macroscpp±tr- level. More recently it is come to emphasize 
the need to conceptualize models of society or significant 
social subsystems and to use the models to identify the vari- 
ables in all parts of the system. As they are concerned 
with the dynamic interaction between model elements and the 
measurement and understanding of change, the indicators must 
be time-series. Once the interrelationships in the model 
have been established empirically, monitoring change in the 
variables becomes a means of anticipating change in other 
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parts of the system. As development of such a model is a very 
long-term goal, it must be approached through a process of 
successive approximations. Even so, a beginning must be made 
(pp. 67ff).'^ 

The statement infers a KPU system that is highly interconnected and quite 
explicit, that is, relationships throughout the system can be clearly 
identified, and knowledge about them can lead to an effective monitoring 
program. Howevfer, several recent studies, including our own, suggest 
that such a position may be overly ambitious at the present time. Further 
more, there are presently a number of conflicting views or conceptualiza- 
tions of an overall system for educational KPU; which perspective is 
adopted has important implications for an NIE-based monitoring program. 

Mechanistic Versus Self-Organizing or Living Systems 

How are we to choose a single perspective that best satisfies both 
the criteria of a "reconstructed logic" (an empirically-based model) and 
the requirements for monitoring? For the most part, we agree with Cuba 
and Clark (1974) where they emphasize a "conf igurational perspective" 
and refute a systems view of educational KPU that is decidedl y mechanistic 
We found that analyses of configurations (based on satisfying particular 
analytical purposes) was far more profitable than describing or mapping 
all educational KPU governance for all reasons. We wish to note, however, 
that there are alternative formulations of systems that largely reject 
mechanistic or "machinelike" interpretations of human organization and 
behavior. Prominent among these are approaches to self- organizing 
(Ashby, in Buckley, 1968), living (Miller, 1965; -Wallace, indraft), 
and adaptive (Bateson, in Vori Foerster et. al., 1974) systems views. 



See E. B. Sheldon and R. Parke, "Social Indicators, " Science , Vol. 188, 
pp. 693-99 (16 May 1975) for a discussion of the state of the art in 
social indicator research. 
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Importantly, all of these emphasize interaction (between *an agent and its 
environment) and, in Bateson's words, "coevolution. " In our view, these 
alternative concepts are largely consistent with a conf igurational image 

JL. 

of educational KPU. Unfortunately their e.pplication to modeling large 
social systems is still somewhat beyond the state of the art; still they 
offer more realistic approaches to social dynamics than do mechanistic 
systems approaches. 

In arguing against a [mechanistic] systems view of educational KPU, 
Cuba and Clark characterize the present interpretation of KPU in educa- 
tion as an admittedly linear systems model; in Table B-1, we reconstruct 
the properties they assign to this perspective. Table B-2 is a summary 
list of properties of living systems versus those of mechanistic systems. 
When compared with Table B-1, Table B-2 indicates that Cuba and Clark's 
criticisms deserve to be leveled at the mechanistic systems view and not 
at the systems view per se. 

Implications for a Monitoring Program 

T-h-e— q-ues^feion-5 — feh^n-^ — i-s— not-whether- the systems view is capable of 

dealing with educational KPU in a realistic, comprehensive, and balanced 
way (as we believe it is); rather we must ask whether the type of systems 
view that could fulfill these requirements could also be used as an 



We hope that in making the above contrasts no reader will infer that 
we disagree with Cuba and Clark's essential conclusions regarding the 
utility of what they have termed the "conf igurational perspective." 
In fact, our analytical framework and findings agree with their charac- 
terization of the nature of educational KPU. We do, however, disagree 
with some of the inferences they draw, for example, that the proper NIE 
response to a constituency that is a "nonsystem" is to delegate much of 
its decision-making responsibility to that constituency. 
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Table B-1 

PROPERTIES OF THE CONFIGURATIONAL PERSPECTIVE AND THE SYSTEMS 
VIEW OF KPU ACCORDING TO CUBA AND CLARK'S (1974) ESSAY 



The Systems View 



The Conf igurational Perspective 



4. 



The basic contention is that the KPU planner would be much 
closer to descrir^cive reality if he were to picture: 



A KPU system 



1. An educational KPU community 
(pp. 29 ff) 



Which can congruently be discussed using 
terminology such as: 



Allocation, authority, compul- 
sion, delegation, assignment.. 



2. Political negotiation, persuasion, 
responsibility, and commitment 
(p. 30) 



And which is true because the view assumes that: 



The functions of R&D are linear 
and can be linked, thereby form- 
ing: 

(a) a system of agencies and 
agents 

(b) with assigned functions and 
responsibilities in RDDA, 

(c) sharing goals, 

(d) and directed to productive 
output which would result 
in improvement oriented 

, change in schools 

(p. 20) 



Few hierarchical relationships with 
authority allocations exist among 
the agencies and agents of educa- 
tional KPU 
(p. 30) 



The root metaphor for ^system 
is, . .mechanical. . .it implies 
some sort of mechanism with a 
variety of parts moving to- 
gether to achieve some common 
end. 
(p. 25) 



Thus: 



4. 



The term "community like" is in- 
tended to identify the root meta- 
phor undergirding the configura- 
tional model in the same sense that 
"machinelike" is the root metaphor 
for the systems view, 
(p. 29) 
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Table B-2 

MECHANISTIC VERSUS LIVING SYSTEMS PARADIGMS 



Machinelike Systems Paradigm 



1. The system must be designed and built. 



2, It must be organized by its leaders 
and managers. 

3, The system Is defined by Its struc- 
ture^ functions^ and boundaries. It 
can be thought of as free- standing^ 
stable^ and real. 

4, It has a rigid and well-defined struc- 
ture. 



5. Parts of the system can be analyzed 
and treated Independently of each 
other and of the whole. 

6. Things happen by chains of linear 
cause and effect. 

7. Relationships tend to be firm until 
they are consciously changed. 

8. The system's behavior has little or no 
unintended side effects on its envi- 
ronment. 

9. Management is exercised mainly through 
command^ direction^ and authority^ 
with emphasis on 

10. Formally defined decisions^ rules^ 
procedures^ and roles. 

11. The emphasis Is on power and author- 
ity. 

12. System hierarchy Is seen as a rigid 
structure of power relationships. 

13. Control Is though of as the key to 
managerial success. 

14. Planning Is seen as a formalized 
decision-making discipline. 

15. Monitoring Is seen as the collection 
of quantitative data that always re- 
flect the same variables. 



Living Systems Paradigm 



1. The system Is evolved and grown in open-ended and 
indeterminate processes^ with some "external" de- 
sign assistance and directfon. 

2. It Is self-organizing and self-adaptive to an Im- 
portant degree, 

3. The system Is seen as fluld^ dynamically stable^ 
always changing In live interaction with its en- 
vironment. It can be seen as a state of con- 
sciousness of its members and others. 

4. A changeable and changing structure exists^ gen- 
erated by and embodying past' processes^ and in- 
fluencing present and future process possibili- 
ties. 

5. Everything interacts dynamically with everything 
else^ and often In counterintuitive ways. 

6. Changes in state result from nonlinear and dyn- 
amically Interactive effects of positive and 
negative feedback. 

7. Relationships are always in a process of change^ 
whether or not we are aware of them. 

8. Great attention must be given to possible side 
effects^ including second and third order. 

9. Management is exercised mainly by responsibility^ 
Influence^ expertise^ creativity^ and process de- 
sign^ with emphasis on 

10. Finding and pursuing goals^ by Initiating open- 
ended processes^ development of a climate and 
ethlc^ role adaptation^ and mutual learning. 

11. The emphasis is on the right Information at the 
right place at the right time. 

12. Hierarchy Is seen In terms of the emergence of 
qualitatively different functions at each level. 

13. Coordinated^ contextual^ responsible autonomy is 
the key to managerial success, 

14. Planning is seen as a mutual learning and 
decision- resolving process. 

15. Monitoring Is seen as the collection of data 
about variables that change as a result of par- 
ticular concern and different context. 
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organizing basis for a monitoring program for NIE. The question cannot 
be answered simply: 

• In general, a linear model of educational KPU (e.g., RDDA) 
is an inaccurate way of characterizing the entire enter- 
prise. Social indicators selected on this basis to track 
change in education are in this sense misleading. There 
may be specific KPU institutions where certain processes 
are designed to be linear. Here, and only in a limited 
fashion, can indicators be used to monitor change. Particu- 
lar care must be taken not to generalize such indicators as 
representative of the whole. 

• Insofar as configurational analyses are performed to answer 
specific questions or address specific issues about educa- 
tional KPU and/ or educational KPU governance, we believe 
monitoring is feasible. 

• Where empirically justified, configurational analyses may. 
become more holistic, that is, larger configurations made 
up of significantly interacting smaller configurations 
with which the present study, has dealt. Where such 
analysis is successful, only then can the feasibility of 

a comprehensive monitoring system be judged. 

To state this another way: within the limits of a particular con- 
figuration, social indicator monitoring may be practical; neither theory 
nor data are as yet sufficient to prescribe general indicators of educa- 
tional KPU conduct. Our own experience, as well as that of Cuba and 
Clark and others doing similar systems analyses of educational KPU, does 
no^t affirm the notion of a tight-knit, logically interconnected, and 
linear educational KPU stem. We do not believe that educational KPU, 
viewed as a whole, should as yet (if ever) be characterized as a "formal* 
system; nor should it be monitored as such. However, applying systems 
principles to mapping configurations is feasible, economical, and im- 
mediately practicable; it should be pursued as an organizing basis for a 
necessarily primitive monitoring program that addresses specific educa- 
tional KPU concerns rather than general areas. 
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